[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 14:58:07 dist-test-slave-68xk krb5kdc[11305](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 14:58:07 dist-test-slave-68xk krb5kdc[11305](info): set up 2 sockets
Aug 06 14:58:07 dist-test-slave-68xk krb5kdc[11305](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 14:58:09.017474 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 14:58:09 dist-test-slave-68xk krb5kdc[11305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T14:58:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T14:58:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 14:58:09.146014 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:39227
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:45085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:39227
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:09.244450 11327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:09.244685 11327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:09.244711 11327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:09.245985 11327 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 14:58:09.246037 11327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:09.246052 11327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:09.246065 11327 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 14:58:09.246078 11327 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 14:58:09.247828 11327 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:45085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:39227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:39227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:09.248167 11327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:09.248405 11327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:09.256078 11334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:09.256227 11333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:09.256690 11337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:09.257121 11327 server_base.cc:1034] running on GCE node
I20260806 14:58:09.257436 11327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:09.257843 11327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:09.259016 11327 hybrid_clock.cc:648] HybridClock initialized: now 1786028289258993 us; error 42 us; skew 500 ppm
Aug 06 14:58:09 dist-test-slave-68xk krb5kdc[11305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028289, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:09.268831 11327 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 14:58:09.269539 11327 webserver.cc:460] Webserver started at http://127.11.8.126:37539/ using document root <none> and password file <none>
I20260806 14:58:09.269882 11327 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:09.269961 11327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:09.270166 11327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:09.271595 11327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "ae3c6e64dcd44d3c9a1d88ac167da208"
format_stamp: "Formatted at 2026-08-06 14:58:09 on dist-test-slave-68xk"
server_key: "e5f9d5d0504b6cef98644b42b81e912f"
server_key_iv: "996a3032597dc5fba552a88b25ae7d85"
server_key_version: "encryptionkey@0"
I20260806 14:58:09.272089 11327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "ae3c6e64dcd44d3c9a1d88ac167da208"
format_stamp: "Formatted at 2026-08-06 14:58:09 on dist-test-slave-68xk"
server_key: "e5f9d5d0504b6cef98644b42b81e912f"
server_key_iv: "996a3032597dc5fba552a88b25ae7d85"
server_key_version: "encryptionkey@0"
I20260806 14:58:09.273912 11327 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 14:58:09.275100 11346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:09.275499 11327 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 14:58:09.275610 11327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "ae3c6e64dcd44d3c9a1d88ac167da208"
format_stamp: "Formatted at 2026-08-06 14:58:09 on dist-test-slave-68xk"
server_key: "e5f9d5d0504b6cef98644b42b81e912f"
server_key_iv: "996a3032597dc5fba552a88b25ae7d85"
server_key_version: "encryptionkey@0"
I20260806 14:58:09.275671 11327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:09.292227 11327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:09.294701 11327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:09.294903 11327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:09.300158 11327 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:39227
I20260806 14:58:09.300259 11424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:39227 every 8 connection(s)
I20260806 14:58:09.300800 11327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 14:58:09.301994 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 11327
I20260806 14:58:09.302080 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 14:58:09.302340 11425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:09.302795 11297 external_mini_cluster.cc:1428] Setting key cfd3fffa7a6146c5b24e61689234bb05
I20260806 14:58:09.306164 11425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae3c6e64dcd44d3c9a1d88ac167da208: Bootstrap starting.
I20260806 14:58:09.307119 11425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae3c6e64dcd44d3c9a1d88ac167da208: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:09.307487 11425 log.cc:824] T 00000000000000000000000000000000 P ae3c6e64dcd44d3c9a1d88ac167da208: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:09.308295 11425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae3c6e64dcd44d3c9a1d88ac167da208: No bootstrap required, opened a new log
I20260806 14:58:09.310727 11425 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae3c6e64dcd44d3c9a1d88ac167da208 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae3c6e64dcd44d3c9a1d88ac167da208" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 39227 } }
I20260806 14:58:09.310952 11425 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae3c6e64dcd44d3c9a1d88ac167da208 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:09.311074 11425 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae3c6e64dcd44d3c9a1d88ac167da208 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae3c6e64dcd44d3c9a1d88ac167da208, State: Initialized, Role: FOLLOWER
I20260806 14:58:09.311260 11425 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae3c6e64dcd44d3c9a1d88ac167da208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae3c6e64dcd44d3c9a1d88ac167da208" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 39227 } }
I20260806 14:58:09.311408 11425 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae3c6e64dcd44d3c9a1d88ac167da208 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 14:58:09.311458 11425 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae3c6e64dcd44d3c9a1d88ac167da208 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 14:58:09.311515 11425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae3c6e64dcd44d3c9a1d88ac167da208 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:09.312071 11425 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae3c6e64dcd44d3c9a1d88ac167da208 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae3c6e64dcd44d3c9a1d88ac167da208" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 39227 } }
I20260806 14:58:09.312304 11425 leader_election.cc:302] T 00000000000000000000000000000000 P ae3c6e64dcd44d3c9a1d88ac167da208 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae3c6e64dcd44d3c9a1d88ac167da208; no voters:
I20260806 14:58:09.312565 11425 leader_election.cc:288] T 00000000000000000000000000000000 P ae3c6e64dcd44d3c9a1d88ac167da208 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 14:58:09.312590 11430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae3c6e64dcd44d3c9a1d88ac167da208 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:09.312954 11425 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae3c6e64dcd44d3c9a1d88ac167da208 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 14:58:09.314358 11430 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae3c6e64dcd44d3c9a1d88ac167da208 [term 1 LEADER]: Becoming Leader. State: Replica: ae3c6e64dcd44d3c9a1d88ac167da208, State: Running, Role: LEADER
I20260806 14:58:09.314571 11430 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae3c6e64dcd44d3c9a1d88ac167da208 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae3c6e64dcd44d3c9a1d88ac167da208" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 39227 } }
I20260806 14:58:09.315150 11446 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae3c6e64dcd44d3c9a1d88ac167da208 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae3c6e64dcd44d3c9a1d88ac167da208. Latest consensus state: current_term: 1 leader_uuid: "ae3c6e64dcd44d3c9a1d88ac167da208" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae3c6e64dcd44d3c9a1d88ac167da208" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 39227 } } }
I20260806 14:58:09.315308 11446 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae3c6e64dcd44d3c9a1d88ac167da208 [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:09.315578 11432 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae3c6e64dcd44d3c9a1d88ac167da208 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae3c6e64dcd44d3c9a1d88ac167da208" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae3c6e64dcd44d3c9a1d88ac167da208" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 39227 } } }
I20260806 14:58:09.315793 11432 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae3c6e64dcd44d3c9a1d88ac167da208 [sys.catalog]: This master's current role is: LEADER
Aug 06 14:58:09 dist-test-slave-68xk krb5kdc[11305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
W20260806 14:58:09.317363 11450 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ae3c6e64dcd44d3c9a1d88ac167da208: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 14:58:09.317442 11450 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 14:58:09.317567 11452 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 14:58:09.317955 11452 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 14:58:09.320501 11452 catalog_manager.cc:1347] Generated new cluster ID: 3d028b363a994939a202f5c8ba4b7fa5
I20260806 14:58:09.320557 11452 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 14:58:09.321981 11427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:09.304226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51888 (local address 127.11.8.126:39227)
0806 14:58:09.304496 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:09.304504 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:09.304831 (+ 327us) server_negotiation.cc:407] Connection header received
0806 14:58:09.307459 (+ 2628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:09.307466 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:09.307577 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:09.307784 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:09.309455 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:09.310697 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:09.312157 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:09.312783 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:09.314092 (+ 1309us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:09.317274 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:09.317278 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:09.317278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:09.317290 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:09.319430 (+ 2140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:09.320101 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:09.320104 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:09.320104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:09.320237 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:09.320671 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:09.320672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:09.320672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:09.321008 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:09.321135 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:09.321592 (+ 457us) server_negotiation.cc:299] Negotiation successful
0806 14:58:09.321759 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":93,"threads_started":1}
I20260806 14:58:09.327852 11452 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 14:58:09.328379 11452 catalog_manager.cc:1504] Loading token signing keys...
I20260806 14:58:09.332336 11452 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae3c6e64dcd44d3c9a1d88ac167da208: Generated new TSK 0
I20260806 14:58:09.332525 11452 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
I20260806 14:58:09.365538 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:0
--local_ip_for_outbound_sockets=127.11.8.65
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:39227
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:45085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:09.493635 11459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:09.494024 11459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:09.494112 11459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:09.495738 11459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:09.495821 11459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:09.495885 11459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 14:58:09.498273 11459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:45085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:39227
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:09.498761 11459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:09.499281 11459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:09.506446 11468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:09.506494 11471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:09.506834 11466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:09.507368 11459 server_base.cc:1034] running on GCE node
I20260806 14:58:09.507565 11459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:09.507853 11459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:09.509038 11459 hybrid_clock.cc:648] HybridClock initialized: now 1786028289509021 us; error 33 us; skew 500 ppm
Aug 06 14:58:09 dist-test-slave-68xk krb5kdc[11305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028289, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:09.512283 11459 init.cc:377] Logged in from keytab as kudu/127.11.8.65@KRBTEST.COM (short username kudu)
I20260806 14:58:09.512907 11459 webserver.cc:460] Webserver started at http://127.11.8.65:38577/ using document root <none> and password file <none>
I20260806 14:58:09.513161 11459 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:09.513244 11459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:09.513448 11459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:09.514926 11459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-0/data/instance:
uuid: "5c9e531e07bf4aacbc7c86572a1853f7"
format_stamp: "Formatted at 2026-08-06 14:58:09 on dist-test-slave-68xk"
server_key: "851e52f2418a552021e5db5008a9de88"
server_key_iv: "4a0cb5ca5f2708ce4293c633f74a3f9e"
server_key_version: "encryptionkey@0"
I20260806 14:58:09.515336 11459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance:
uuid: "5c9e531e07bf4aacbc7c86572a1853f7"
format_stamp: "Formatted at 2026-08-06 14:58:09 on dist-test-slave-68xk"
server_key: "851e52f2418a552021e5db5008a9de88"
server_key_iv: "4a0cb5ca5f2708ce4293c633f74a3f9e"
server_key_version: "encryptionkey@0"
I20260806 14:58:09.516839 11459 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 14:58:09.517722 11480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:09.518049 11459 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:09.518170 11459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "5c9e531e07bf4aacbc7c86572a1853f7"
format_stamp: "Formatted at 2026-08-06 14:58:09 on dist-test-slave-68xk"
server_key: "851e52f2418a552021e5db5008a9de88"
server_key_iv: "4a0cb5ca5f2708ce4293c633f74a3f9e"
server_key_version: "encryptionkey@0"
I20260806 14:58:09.518247 11459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:09.533198 11459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:09.535838 11459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:09.536073 11459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:09.536556 11459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:09.537003 11459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:09.537106 11459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:09.537144 11459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:09.537194 11459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:09.550559 11459 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:40395
I20260806 14:58:09.550645 11636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:40395 every 8 connection(s)
I20260806 14:58:09.550992 11459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
I20260806 14:58:09.552346 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 11459
I20260806 14:58:09.552451 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance
I20260806 14:58:09.552646 11297 external_mini_cluster.cc:1428] Setting key af3478d86ba07f0a0bcff17a2283f4a2
Aug 06 14:58:09 dist-test-slave-68xk krb5kdc[11305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028289, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.8.66@KRBTEST.COM; defaulting to no policy
I20260806 14:58:09.561625 11427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:09.551999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:43843 (local address 127.11.8.126:39227)
0806 14:58:09.552045 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:09.552048 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:09.552477 (+ 429us) server_negotiation.cc:407] Connection header received
0806 14:58:09.553186 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:09.553187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:09.553200 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:09.553243 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:09.554480 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:09.554990 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:09.556034 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:09.556265 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:09.556352 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:09.558728 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:09.558730 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:09.558730 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:09.558737 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:09.560192 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:09.560621 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:09.560623 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:09.560623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:09.560686 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:09.560955 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:09.560956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:09.560956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:09.561074 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:09.561151 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:09.561434 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 14:58:09.561528 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 14:58:09.561820 11642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:09.551974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:39227 (local address 127.11.8.65:43843)
0806 14:58:09.552304 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:09.552344 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:09.553070 (+ 726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:09.553341 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:09.553345 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:09.554144 (+ 799us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:09.554351 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:09.554356 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:09.555098 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:09.555100 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:09.555929 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:09.555932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:09.556364 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:09.556365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:09.556507 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:09.557088 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:09.557097 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:09.558630 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:09.560297 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:09.560305 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:09.560307 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:09.560542 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:09.560763 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:09.560764 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:09.560764 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:09.560879 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:09.561140 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:09.561147 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:09.561301 (+ 154us) client_negotiation.cc:769] Sending connection context
0806 14:58:09.561414 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 14:58:09.561583 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":214,"threads_started":1}
I20260806 14:58:09.562695 11638 heartbeater.cc:347] Connected to a master server at 127.11.8.126:39227
I20260806 14:58:09.562831 11638 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:09.563035 11638 heartbeater.cc:511] Master 127.11.8.126:39227 requested a full tablet report, sending...
I20260806 14:58:09.564112 11371 ts_manager.cc:194] Registered new tserver with Master: 5c9e531e07bf4aacbc7c86572a1853f7 (127.11.8.65:40395)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.66@KRBTEST.COM" created.
I20260806 14:58:09.565089 11371 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.65@KRBTEST.COM'} at 127.11.8.65:43843
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
I20260806 14:58:09.581845 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:0
--local_ip_for_outbound_sockets=127.11.8.66
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:39227
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:45085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:09.665961 11646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:09.666162 11646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:09.666186 11646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:09.667474 11646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:09.667520 11646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:09.667547 11646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 14:58:09.668941 11646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:45085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:39227
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:09.669119 11646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:09.669303 11646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:09.674584 11656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:09.674642 11646 server_base.cc:1034] running on GCE node
W20260806 14:58:09.674566 11661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:09.674613 11658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:09.675069 11646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:09.675302 11646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:09.676504 11646 hybrid_clock.cc:648] HybridClock initialized: now 1786028289676503 us; error 42 us; skew 500 ppm
Aug 06 14:58:09 dist-test-slave-68xk krb5kdc[11305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028289, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:09.679646 11646 init.cc:377] Logged in from keytab as kudu/127.11.8.66@KRBTEST.COM (short username kudu)
I20260806 14:58:09.680172 11646 webserver.cc:460] Webserver started at http://127.11.8.66:42601/ using document root <none> and password file <none>
I20260806 14:58:09.680398 11646 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:09.680445 11646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:09.680542 11646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:09.681630 11646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-1/data/instance:
uuid: "6c7c16bf5ca84bb59f2c30ce87905157"
format_stamp: "Formatted at 2026-08-06 14:58:09 on dist-test-slave-68xk"
server_key: "ce4c3befd2de8dd3929135f2b85d7ba8"
server_key_iv: "49530d27e92d18943872efd2a992eb99"
server_key_version: "encryptionkey@0"
I20260806 14:58:09.681964 11646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance:
uuid: "6c7c16bf5ca84bb59f2c30ce87905157"
format_stamp: "Formatted at 2026-08-06 14:58:09 on dist-test-slave-68xk"
server_key: "ce4c3befd2de8dd3929135f2b85d7ba8"
server_key_iv: "49530d27e92d18943872efd2a992eb99"
server_key_version: "encryptionkey@0"
I20260806 14:58:09.683426 11646 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 14:58:09.684167 11668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:09.684370 11646 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 14:58:09.684486 11646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "6c7c16bf5ca84bb59f2c30ce87905157"
format_stamp: "Formatted at 2026-08-06 14:58:09 on dist-test-slave-68xk"
server_key: "ce4c3befd2de8dd3929135f2b85d7ba8"
server_key_iv: "49530d27e92d18943872efd2a992eb99"
server_key_version: "encryptionkey@0"
I20260806 14:58:09.684573 11646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:09.695731 11646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:09.698248 11646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:09.698477 11646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:09.698748 11646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:09.699095 11646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:09.699156 11646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:09.699201 11646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:09.699231 11646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:09.706869 11646 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:42463
I20260806 14:58:09.706910 11828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:42463 every 8 connection(s)
I20260806 14:58:09.707276 11646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
Aug 06 14:58:09 dist-test-slave-68xk krb5kdc[11305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028289, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:09.716809 11427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:09.708138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:55323 (local address 127.11.8.126:39227)
0806 14:58:09.708228 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:09.708230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:09.708431 (+ 201us) server_negotiation.cc:407] Connection header received
0806 14:58:09.708978 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:09.708980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:09.708992 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:09.709055 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:09.709995 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:09.710571 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:09.711579 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:09.711791 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:09.711869 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:09.714301 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:09.714302 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:09.714302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:09.714309 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:09.715581 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:09.715985 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:09.715986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:09.715986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:09.716040 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:09.716281 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:09.716282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:09.716282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:09.716386 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:09.716438 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:09.716671 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 14:58:09.716742 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 14:58:09.716970 11833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:09.708092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:39227 (local address 127.11.8.66:55323)
0806 14:58:09.708293 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:09.708339 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:09.708867 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:09.709125 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:09.709134 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:09.709688 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:09.709888 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:09.709893 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:09.710664 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:09.710665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:09.711474 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:09.711476 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:09.711874 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:09.711875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:09.712000 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:09.712559 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:09.712568 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:09.714198 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:09.715660 (+ 1462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:09.715668 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:09.715670 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:09.715903 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:09.716105 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:09.716106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:09.716106 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:09.716209 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:09.716444 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:09.716448 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:09.716563 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 14:58:09.716644 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 14:58:09.716771 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":105,"threads_started":1}
I20260806 14:58:09.717480 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 11646
I20260806 14:58:09.717516 11831 heartbeater.cc:347] Connected to a master server at 127.11.8.126:39227
I20260806 14:58:09.717592 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance
I20260806 14:58:09.717630 11831 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:09.717783 11297 external_mini_cluster.cc:1428] Setting key e46611c5f8f4a7f9b8bb1fd892775182
I20260806 14:58:09.717856 11831 heartbeater.cc:511] Master 127.11.8.126:39227 requested a full tablet report, sending...
I20260806 14:58:09.718257 11369 ts_manager.cc:194] Registered new tserver with Master: 6c7c16bf5ca84bb59f2c30ce87905157 (127.11.8.66:42463)
I20260806 14:58:09.718819 11369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.66@KRBTEST.COM'} at 127.11.8.66:55323
WARNING: no policy specified for kudu/127.11.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
I20260806 14:58:09.749291 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:0
--local_ip_for_outbound_sockets=127.11.8.67
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:39227
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:45085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:09.833722 11837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:09.833969 11837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:09.834008 11837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:09.835242 11837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:09.835317 11837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:09.835363 11837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 14:58:09.837178 11837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:45085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:39227
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:09.837477 11837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:09.837723 11837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:09.843230 11850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:09.843201 11846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:09.843343 11837 server_base.cc:1034] running on GCE node
W20260806 14:58:09.843201 11847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:09.843600 11837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:09.843807 11837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:09.844946 11837 hybrid_clock.cc:648] HybridClock initialized: now 1786028289844932 us; error 30 us; skew 500 ppm
Aug 06 14:58:09 dist-test-slave-68xk krb5kdc[11305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028289, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:09.847512 11837 init.cc:377] Logged in from keytab as kudu/127.11.8.67@KRBTEST.COM (short username kudu)
I20260806 14:58:09.847980 11837 webserver.cc:460] Webserver started at http://127.11.8.67:42659/ using document root <none> and password file <none>
I20260806 14:58:09.848230 11837 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:09.848276 11837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:09.848448 11837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:09.849484 11837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-2/data/instance:
uuid: "97e653dabd6a4280b7a7909691d2d530"
format_stamp: "Formatted at 2026-08-06 14:58:09 on dist-test-slave-68xk"
server_key: "4e03c48e5f6ba2b646b155c6697fc6d4"
server_key_iv: "ee1a227b166ab7260944320c0349f3d3"
server_key_version: "encryptionkey@0"
I20260806 14:58:09.849867 11837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance:
uuid: "97e653dabd6a4280b7a7909691d2d530"
format_stamp: "Formatted at 2026-08-06 14:58:09 on dist-test-slave-68xk"
server_key: "4e03c48e5f6ba2b646b155c6697fc6d4"
server_key_iv: "ee1a227b166ab7260944320c0349f3d3"
server_key_version: "encryptionkey@0"
I20260806 14:58:09.851311 11837 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 14:58:09.852093 11859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:09.852279 11837 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 14:58:09.852376 11837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "97e653dabd6a4280b7a7909691d2d530"
format_stamp: "Formatted at 2026-08-06 14:58:09 on dist-test-slave-68xk"
server_key: "4e03c48e5f6ba2b646b155c6697fc6d4"
server_key_iv: "ee1a227b166ab7260944320c0349f3d3"
server_key_version: "encryptionkey@0"
I20260806 14:58:09.852449 11837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:09.866185 11837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:09.868209 11837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:09.868368 11837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:09.868590 11837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:09.868948 11837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:09.869004 11837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:09.869046 11837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:09.869074 11837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:09.876690 11837 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:40111
I20260806 14:58:09.876736 12008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:40111 every 8 connection(s)
I20260806 14:58:09.877044 11837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
Aug 06 14:58:09 dist-test-slave-68xk krb5kdc[11305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028289, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:09.884789 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 11837
I20260806 14:58:09.884881 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance
I20260806 14:58:09.885047 11297 external_mini_cluster.cc:1428] Setting key 6429eea47541889c6c9b7fec4355ecfe
I20260806 14:58:09.886940 11427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:09.877843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:48799 (local address 127.11.8.126:39227)
0806 14:58:09.877924 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:09.877926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:09.878105 (+ 179us) server_negotiation.cc:407] Connection header received
0806 14:58:09.878694 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:09.878695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:09.878706 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:09.878760 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:09.879869 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:09.880372 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:09.881344 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:09.881525 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:09.881577 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:09.883651 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:09.883653 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:09.883653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:09.883660 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:09.885269 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:09.885836 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:09.885838 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:09.885838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:09.885901 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:09.886289 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:09.886290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:09.886291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:09.886471 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:09.886525 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:09.886810 (+ 285us) server_negotiation.cc:299] Negotiation successful
0806 14:58:09.886877 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 14:58:09.887239 12013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:09.877792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:39227 (local address 127.11.8.67:48799)
0806 14:58:09.877976 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:09.878012 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:09.878559 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:09.878836 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:09.878839 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:09.879544 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:09.879780 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:09.879785 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:09.880488 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:09.880489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:09.881271 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:09.881273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:09.881602 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:09.881603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:09.881704 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:09.882143 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:09.882151 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:09.883564 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:09.885373 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:09.885381 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:09.885383 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:09.885750 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:09.885996 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:09.885998 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:09.885998 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:09.886200 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:09.886524 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:09.886531 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:09.886682 (+ 151us) client_negotiation.cc:769] Sending connection context
0806 14:58:09.886779 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 14:58:09.886911 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":99,"threads_started":1}
I20260806 14:58:09.887874 12009 heartbeater.cc:347] Connected to a master server at 127.11.8.126:39227
I20260806 14:58:09.887991 12009 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:09.888190 12009 heartbeater.cc:511] Master 127.11.8.126:39227 requested a full tablet report, sending...
I20260806 14:58:09.888568 11369 ts_manager.cc:194] Registered new tserver with Master: 97e653dabd6a4280b7a7909691d2d530 (127.11.8.67:40111)
I20260806 14:58:09.888944 11369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.67@KRBTEST.COM'} at 127.11.8.67:48799
I20260806 14:58:09.897478 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 14:58:09 dist-test-slave-68xk krb5kdc[11305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028289, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 14:58:09 dist-test-slave-68xk krb5kdc[11305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028289, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.8.65@KRBTEST.COM
I20260806 14:58:09.923389 12021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:09.915380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50678 (local address 127.11.8.65:40395)
0806 14:58:09.915576 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:09.915582 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:09.915593 (+ 11us) server_negotiation.cc:407] Connection header received
0806 14:58:09.915623 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:09.915626 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:09.915640 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:09.915710 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:09.916287 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:09.917008 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:09.917743 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:09.918047 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:09.918103 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:09.920556 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:09.920565 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:09.920565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:09.920573 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:09.922152 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:09.922534 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:09.922537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:09.922538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:09.922599 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:09.922814 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:09.922815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:09.922815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:09.922950 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:09.923058 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:09.923122 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 14:58:09.923191 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":95,"threads_started":1}
W20260806 14:58:09.923890 11515 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50678
Aug 06 14:58:09 dist-test-slave-68xk krb5kdc[11305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 14:58:09 dist-test-slave-68xk krb5kdc[11305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.65@KRBTEST.COM
I20260806 14:58:09.948112 12021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:09.940597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50686 (local address 127.11.8.65:40395)
0806 14:58:09.940670 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:09.940671 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:09.940740 (+ 69us) server_negotiation.cc:407] Connection header received
0806 14:58:09.940800 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:09.940800 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:09.940814 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:09.940865 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:09.941495 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:09.941966 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:09.942777 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:09.942979 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:09.943039 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:09.945407 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:09.945409 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:09.945409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:09.945416 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:09.946929 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:09.947317 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:09.947318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:09.947318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:09.947367 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:09.947590 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:09.947591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:09.947591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:09.947710 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:09.947817 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:09.947966 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 14:58:09.948039 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 14:58:09.948719 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 11459
I20260806 14:58:09.962110 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 11646
I20260806 14:58:09.968802 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 11837
I20260806 14:58:09.976013 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 11327
2026-08-06T14:58:09Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2238 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 14:58:10 dist-test-slave-68xk krb5kdc[12028](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 14:58:10 dist-test-slave-68xk krb5kdc[12028](info): set up 2 sockets
Aug 06 14:58:10 dist-test-slave-68xk krb5kdc[12028](info): commencing operation
krb5kdc: starting...
W20260806 14:58:11.174711 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 14:58:11 dist-test-slave-68xk krb5kdc[12028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T14:58:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T14:58:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 14:58:11.301287 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:39037
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:37231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:39037
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:11.383011 12045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:11.383203 12045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:11.383225 12045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:11.384346 12045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 14:58:11.384388 12045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:11.384403 12045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:11.384452 12045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 14:58:11.384471 12045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 14:58:11.386022 12045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:37231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:39037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:39037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:11.386268 12045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:11.386539 12045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:11.391738 12050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:11.392793 12045 server_base.cc:1034] running on GCE node
W20260806 14:58:11.392952 12051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:11.393271 12055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:11.393536 12045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:11.393842 12045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:11.395008 12045 hybrid_clock.cc:648] HybridClock initialized: now 1786028291394999 us; error 51 us; skew 500 ppm
Aug 06 14:58:11 dist-test-slave-68xk krb5kdc[12028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028291, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:11.397450 12045 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 14:58:11.397938 12045 webserver.cc:460] Webserver started at http://127.11.8.126:45413/ using document root <none> and password file <none>
I20260806 14:58:11.398191 12045 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:11.398269 12045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:11.398450 12045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:11.399482 12045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "d353480994ac409289fe4b94cddeb3ed"
format_stamp: "Formatted at 2026-08-06 14:58:11 on dist-test-slave-68xk"
server_key: "3b05d7287cd75f0945609f6a1d1c883d"
server_key_iv: "d49165d8eb69f2eeb83d9fba7ba3a851"
server_key_version: "encryptionkey@0"
I20260806 14:58:11.399893 12045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "d353480994ac409289fe4b94cddeb3ed"
format_stamp: "Formatted at 2026-08-06 14:58:11 on dist-test-slave-68xk"
server_key: "3b05d7287cd75f0945609f6a1d1c883d"
server_key_iv: "d49165d8eb69f2eeb83d9fba7ba3a851"
server_key_version: "encryptionkey@0"
I20260806 14:58:11.401238 12045 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 14:58:11.402000 12065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:11.402182 12045 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 14:58:11.402305 12045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "d353480994ac409289fe4b94cddeb3ed"
format_stamp: "Formatted at 2026-08-06 14:58:11 on dist-test-slave-68xk"
server_key: "3b05d7287cd75f0945609f6a1d1c883d"
server_key_iv: "d49165d8eb69f2eeb83d9fba7ba3a851"
server_key_version: "encryptionkey@0"
I20260806 14:58:11.402381 12045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:11.412119 12045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:11.413966 12045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:11.414135 12045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:11.418282 12045 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:39037
I20260806 14:58:11.418383 12136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:39037 every 8 connection(s)
I20260806 14:58:11.418754 12045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 14:58:11.419667 12137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:11.421886 12137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d353480994ac409289fe4b94cddeb3ed: Bootstrap starting.
I20260806 14:58:11.422533 12137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d353480994ac409289fe4b94cddeb3ed: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:11.422828 12137 log.cc:824] T 00000000000000000000000000000000 P d353480994ac409289fe4b94cddeb3ed: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:11.423529 12137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d353480994ac409289fe4b94cddeb3ed: No bootstrap required, opened a new log
I20260806 14:58:11.425096 12137 raft_consensus.cc:348] T 00000000000000000000000000000000 P d353480994ac409289fe4b94cddeb3ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d353480994ac409289fe4b94cddeb3ed" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 39037 } }
I20260806 14:58:11.425256 12137 raft_consensus.cc:374] T 00000000000000000000000000000000 P d353480994ac409289fe4b94cddeb3ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:11.425314 12137 raft_consensus.cc:729] T 00000000000000000000000000000000 P d353480994ac409289fe4b94cddeb3ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d353480994ac409289fe4b94cddeb3ed, State: Initialized, Role: FOLLOWER
I20260806 14:58:11.425463 12137 consensus_queue.cc:260] T 00000000000000000000000000000000 P d353480994ac409289fe4b94cddeb3ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d353480994ac409289fe4b94cddeb3ed" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 39037 } }
I20260806 14:58:11.425552 12137 raft_consensus.cc:388] T 00000000000000000000000000000000 P d353480994ac409289fe4b94cddeb3ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 14:58:11.425593 12137 raft_consensus.cc:482] T 00000000000000000000000000000000 P d353480994ac409289fe4b94cddeb3ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 14:58:11.425642 12137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d353480994ac409289fe4b94cddeb3ed [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:11.426162 12137 raft_consensus.cc:504] T 00000000000000000000000000000000 P d353480994ac409289fe4b94cddeb3ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d353480994ac409289fe4b94cddeb3ed" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 39037 } }
I20260806 14:58:11.426280 12137 leader_election.cc:302] T 00000000000000000000000000000000 P d353480994ac409289fe4b94cddeb3ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d353480994ac409289fe4b94cddeb3ed; no voters:
I20260806 14:58:11.426367 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 12045
I20260806 14:58:11.426465 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 14:58:11.426481 12137 leader_election.cc:288] T 00000000000000000000000000000000 P d353480994ac409289fe4b94cddeb3ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 14:58:11.426513 12143 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d353480994ac409289fe4b94cddeb3ed [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:11.426663 11297 external_mini_cluster.cc:1428] Setting key 112ffd0256fd75236f4ab5403736a217
I20260806 14:58:11.426664 12143 raft_consensus.cc:686] T 00000000000000000000000000000000 P d353480994ac409289fe4b94cddeb3ed [term 1 LEADER]: Becoming Leader. State: Replica: d353480994ac409289fe4b94cddeb3ed, State: Running, Role: LEADER
I20260806 14:58:11.426769 12143 consensus_queue.cc:237] T 00000000000000000000000000000000 P d353480994ac409289fe4b94cddeb3ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d353480994ac409289fe4b94cddeb3ed" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 39037 } }
I20260806 14:58:11.426782 12137 sys_catalog.cc:564] T 00000000000000000000000000000000 P d353480994ac409289fe4b94cddeb3ed [sys.catalog]: configured and running, proceeding with master startup.
I20260806 14:58:11.427645 12145 sys_catalog.cc:455] T 00000000000000000000000000000000 P d353480994ac409289fe4b94cddeb3ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader d353480994ac409289fe4b94cddeb3ed. Latest consensus state: current_term: 1 leader_uuid: "d353480994ac409289fe4b94cddeb3ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d353480994ac409289fe4b94cddeb3ed" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 39037 } } }
I20260806 14:58:11.427732 12145 sys_catalog.cc:458] T 00000000000000000000000000000000 P d353480994ac409289fe4b94cddeb3ed [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:11.427981 12159 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 14:58:11.427959 12144 sys_catalog.cc:455] T 00000000000000000000000000000000 P d353480994ac409289fe4b94cddeb3ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d353480994ac409289fe4b94cddeb3ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d353480994ac409289fe4b94cddeb3ed" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 39037 } } }
I20260806 14:58:11.428133 12144 sys_catalog.cc:458] T 00000000000000000000000000000000 P d353480994ac409289fe4b94cddeb3ed [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:11.428648 12159 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 14:58:11.430822 12159 catalog_manager.cc:1347] Generated new cluster ID: e453c841db654e2888a04d18f2c5ebee
I20260806 14:58:11.430878 12159 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 14:58:11 dist-test-slave-68xk krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:11.436655 12159 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 14:58:11.437163 12159 catalog_manager.cc:1504] Loading token signing keys...
I20260806 14:58:11.439077 12152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:11.427465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60602 (local address 127.11.8.126:39037)
0806 14:58:11.427844 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:11.427856 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:11.427891 (+ 35us) server_negotiation.cc:407] Connection header received
0806 14:58:11.428006 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:11.428014 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:11.428093 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:11.429295 (+ 1202us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:11.429387 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:11.430758 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:11.431898 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:11.432539 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:11.432664 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:11.435329 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:11.435333 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:11.435334 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:11.435346 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:11.437429 (+ 2083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:11.437836 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:11.437839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:11.437840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:11.437955 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:11.438161 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:11.438161 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:11.438162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:11.438459 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:11.438610 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:11.438719 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 14:58:11.438868 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":71,"threads_started":1}
I20260806 14:58:11.443863 12159 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d353480994ac409289fe4b94cddeb3ed: Generated new TSK 0
I20260806 14:58:11.444025 12159 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
I20260806 14:58:11.476622 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:0
--local_ip_for_outbound_sockets=127.11.8.65
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:39037
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:37231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:11.560863 12176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:11.561100 12176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:11.561144 12176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:11.562312 12176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:11.562379 12176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:11.562445 12176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 14:58:11.563963 12176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:37231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:39037
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:11.564240 12176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:11.564474 12176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:11.569677 12186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:11.569684 12184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:11.569689 12182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:11.570010 12176 server_base.cc:1034] running on GCE node
I20260806 14:58:11.570266 12176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:11.570598 12176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:11.571771 12176 hybrid_clock.cc:648] HybridClock initialized: now 1786028291571762 us; error 29 us; skew 500 ppm
Aug 06 14:58:11 dist-test-slave-68xk krb5kdc[12028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028291, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:11.574501 12176 init.cc:377] Logged in from keytab as kudu/127.11.8.65@KRBTEST.COM (short username kudu)
I20260806 14:58:11.574882 12176 webserver.cc:460] Webserver started at http://127.11.8.65:39001/ using document root <none> and password file <none>
I20260806 14:58:11.575127 12176 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:11.575173 12176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:11.575350 12176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:11.576395 12176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-0/data/instance:
uuid: "75cec4867eaa4f18adec9a71c3250282"
format_stamp: "Formatted at 2026-08-06 14:58:11 on dist-test-slave-68xk"
server_key: "12edeafd1565b826d6a8bd140f1742c4"
server_key_iv: "320294e3c0a31d67d277bdefdf5c61c8"
server_key_version: "encryptionkey@0"
I20260806 14:58:11.576749 12176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance:
uuid: "75cec4867eaa4f18adec9a71c3250282"
format_stamp: "Formatted at 2026-08-06 14:58:11 on dist-test-slave-68xk"
server_key: "12edeafd1565b826d6a8bd140f1742c4"
server_key_iv: "320294e3c0a31d67d277bdefdf5c61c8"
server_key_version: "encryptionkey@0"
I20260806 14:58:11.578128 12176 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 14:58:11.578987 12198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:11.579167 12176 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 14:58:11.579283 12176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "75cec4867eaa4f18adec9a71c3250282"
format_stamp: "Formatted at 2026-08-06 14:58:11 on dist-test-slave-68xk"
server_key: "12edeafd1565b826d6a8bd140f1742c4"
server_key_iv: "320294e3c0a31d67d277bdefdf5c61c8"
server_key_version: "encryptionkey@0"
I20260806 14:58:11.579356 12176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:11.585846 12176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:11.588176 12176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:11.588337 12176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:11.588548 12176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:11.588843 12176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:11.588877 12176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:11.588898 12176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:11.588944 12176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:11.594889 12176 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:36947
I20260806 14:58:11.594933 12351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:36947 every 8 connection(s)
I20260806 14:58:11.595245 12176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
Aug 06 14:58:11 dist-test-slave-68xk krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028291, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:11.601867 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 12176
I20260806 14:58:11.601958 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance
I20260806 14:58:11.602155 11297 external_mini_cluster.cc:1428] Setting key 38c7c0d73f4f920cfc82973e253d68ee
I20260806 14:58:11.605046 12152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:11.595935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:60841 (local address 127.11.8.126:39037)
0806 14:58:11.595980 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:11.595982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:11.596179 (+ 197us) server_negotiation.cc:407] Connection header received
0806 14:58:11.597180 (+ 1001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:11.597181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:11.597190 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:11.597249 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:11.597831 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:11.598347 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:11.599379 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:11.599579 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:11.599636 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:11.601835 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:11.601836 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:11.601836 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:11.601842 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:11.603461 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:11.603958 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:11.603959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:11.603959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:11.604009 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:11.604290 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:11.604290 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:11.604291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:11.604386 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:11.604436 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:11.604799 (+ 363us) server_negotiation.cc:299] Negotiation successful
0806 14:58:11.604905 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 14:58:11.605237 12356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:11.595892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:39037 (local address 127.11.8.65:60841)
0806 14:58:11.596047 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:11.596087 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:11.597088 (+ 1001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:11.597313 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:11.597316 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:11.597558 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:11.597749 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:11.597752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:11.598488 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:11.598489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:11.599299 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:11.599301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:11.599654 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:11.599655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:11.599752 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:11.600272 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:11.600280 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:11.601740 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:11.603547 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:11.603554 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:11.603556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:11.603881 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:11.604074 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:11.604075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:11.604076 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:11.604226 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:11.604449 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:11.604455 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:11.604635 (+ 180us) client_negotiation.cc:769] Sending connection context
0806 14:58:11.604761 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 14:58:11.604980 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260806 14:58:11.605708 12352 heartbeater.cc:347] Connected to a master server at 127.11.8.126:39037
I20260806 14:58:11.605800 12352 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:11.606027 12352 heartbeater.cc:511] Master 127.11.8.126:39037 requested a full tablet report, sending...
I20260806 14:58:11.606633 12089 ts_manager.cc:194] Registered new tserver with Master: 75cec4867eaa4f18adec9a71c3250282 (127.11.8.65:36947)
I20260806 14:58:11.607615 12089 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.65@KRBTEST.COM'} at 127.11.8.65:60841
WARNING: no policy specified for kudu/127.11.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
I20260806 14:58:11.629478 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:0
--local_ip_for_outbound_sockets=127.11.8.66
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:39037
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:37231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:11.711889 12362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:11.712092 12362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:11.712116 12362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:11.713187 12362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:11.713225 12362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:11.713251 12362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 14:58:11.714702 12362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:37231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:39037
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:11.714896 12362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:11.715075 12362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:11.720225 12368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:11.720202 12372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:11.720214 12369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:11.720571 12362 server_base.cc:1034] running on GCE node
I20260806 14:58:11.720880 12362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:11.721158 12362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:11.722325 12362 hybrid_clock.cc:648] HybridClock initialized: now 1786028291722305 us; error 37 us; skew 500 ppm
Aug 06 14:58:11 dist-test-slave-68xk krb5kdc[12028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028291, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:11.724803 12362 init.cc:377] Logged in from keytab as kudu/127.11.8.66@KRBTEST.COM (short username kudu)
I20260806 14:58:11.725287 12362 webserver.cc:460] Webserver started at http://127.11.8.66:41591/ using document root <none> and password file <none>
I20260806 14:58:11.725543 12362 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:11.725615 12362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:11.725771 12362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:11.726948 12362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-1/data/instance:
uuid: "670a0ce3d113488bbf034baef2c8418f"
format_stamp: "Formatted at 2026-08-06 14:58:11 on dist-test-slave-68xk"
server_key: "086e4e4da9441c6dd230300ab67684a4"
server_key_iv: "83836c060adc32612550e69acd92e738"
server_key_version: "encryptionkey@0"
I20260806 14:58:11.727417 12362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance:
uuid: "670a0ce3d113488bbf034baef2c8418f"
format_stamp: "Formatted at 2026-08-06 14:58:11 on dist-test-slave-68xk"
server_key: "086e4e4da9441c6dd230300ab67684a4"
server_key_iv: "83836c060adc32612550e69acd92e738"
server_key_version: "encryptionkey@0"
I20260806 14:58:11.728920 12362 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 14:58:11.729672 12382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:11.729867 12362 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 14:58:11.729972 12362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "670a0ce3d113488bbf034baef2c8418f"
format_stamp: "Formatted at 2026-08-06 14:58:11 on dist-test-slave-68xk"
server_key: "086e4e4da9441c6dd230300ab67684a4"
server_key_iv: "83836c060adc32612550e69acd92e738"
server_key_version: "encryptionkey@0"
I20260806 14:58:11.730062 12362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:11.738346 12362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:11.740271 12362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:11.740439 12362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:11.740669 12362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:11.740996 12362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:11.741055 12362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:11.741096 12362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:11.741123 12362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:11.747593 12362 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:37561
I20260806 14:58:11.747648 12545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:37561 every 8 connection(s)
I20260806 14:58:11.747999 12362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
Aug 06 14:58:11 dist-test-slave-68xk krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028291, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:11.754535 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 12362
I20260806 14:58:11.754639 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance
I20260806 14:58:11.754835 11297 external_mini_cluster.cc:1428] Setting key 22446467836e3647f81a1a209c5cae8e
I20260806 14:58:11.757400 12152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:11.748759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:49585 (local address 127.11.8.126:39037)
0806 14:58:11.748805 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:11.748807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:11.749010 (+ 203us) server_negotiation.cc:407] Connection header received
0806 14:58:11.749938 (+ 928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:11.749939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:11.749950 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:11.750012 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:11.750656 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:11.751150 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:11.752123 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:11.752306 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:11.752358 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:11.754357 (+ 1999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:11.754359 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:11.754359 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:11.754368 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:11.755985 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:11.756395 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:11.756396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:11.756397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:11.756464 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:11.756720 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:11.756721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:11.756722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:11.756844 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:11.756918 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:11.757207 (+ 289us) server_negotiation.cc:299] Negotiation successful
0806 14:58:11.757310 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 14:58:11.757695 12549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:11.748732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:39037 (local address 127.11.8.66:49585)
0806 14:58:11.748886 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:11.748920 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:11.749812 (+ 892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:11.750065 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:11.750068 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:11.750313 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:11.750571 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:11.750577 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:11.751234 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:11.751235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:11.752049 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:11.752051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:11.752379 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:11.752380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:11.752475 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:11.752933 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:11.752941 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:11.754260 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:11.756070 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:11.756077 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:11.756080 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:11.756312 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:11.756541 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:11.756541 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:11.756542 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:11.756651 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:11.756923 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:11.756930 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:11.757068 (+ 138us) client_negotiation.cc:769] Sending connection context
0806 14:58:11.757174 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 14:58:11.757331 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260806 14:58:11.758342 12547 heartbeater.cc:347] Connected to a master server at 127.11.8.126:39037
I20260806 14:58:11.758474 12547 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:11.758675 12547 heartbeater.cc:511] Master 127.11.8.126:39037 requested a full tablet report, sending...
I20260806 14:58:11.759106 12089 ts_manager.cc:194] Registered new tserver with Master: 670a0ce3d113488bbf034baef2c8418f (127.11.8.66:37561)
I20260806 14:58:11.759501 12089 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.66@KRBTEST.COM'} at 127.11.8.66:49585
WARNING: no policy specified for kudu/127.11.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
I20260806 14:58:11.782195 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:0
--local_ip_for_outbound_sockets=127.11.8.67
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:39037
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:37231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:11.865660 12552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:11.865864 12552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:11.865888 12552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:11.866984 12552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:11.867019 12552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:11.867045 12552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 14:58:11.868569 12552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:37231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:39037
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:11.868757 12552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:11.868923 12552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:11.873998 12563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:11.873998 12560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:11.874138 12552 server_base.cc:1034] running on GCE node
W20260806 14:58:11.874229 12561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:11.874523 12552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:11.874782 12552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:11.875941 12552 hybrid_clock.cc:648] HybridClock initialized: now 1786028291875937 us; error 36 us; skew 500 ppm
Aug 06 14:58:11 dist-test-slave-68xk krb5kdc[12028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028291, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:11.878374 12552 init.cc:377] Logged in from keytab as kudu/127.11.8.67@KRBTEST.COM (short username kudu)
I20260806 14:58:11.878911 12552 webserver.cc:460] Webserver started at http://127.11.8.67:38819/ using document root <none> and password file <none>
I20260806 14:58:11.879150 12552 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:11.879221 12552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:11.879354 12552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:11.880374 12552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-2/data/instance:
uuid: "0f542216756d46e991fe5321acd35942"
format_stamp: "Formatted at 2026-08-06 14:58:11 on dist-test-slave-68xk"
server_key: "595a14623123d4a091aa10a18a377cc3"
server_key_iv: "7202d169f76495e523c7bd2912e16487"
server_key_version: "encryptionkey@0"
I20260806 14:58:11.880760 12552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance:
uuid: "0f542216756d46e991fe5321acd35942"
format_stamp: "Formatted at 2026-08-06 14:58:11 on dist-test-slave-68xk"
server_key: "595a14623123d4a091aa10a18a377cc3"
server_key_iv: "7202d169f76495e523c7bd2912e16487"
server_key_version: "encryptionkey@0"
I20260806 14:58:11.882179 12552 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 14:58:11.883069 12572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:11.883240 12552 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 14:58:11.883354 12552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "0f542216756d46e991fe5321acd35942"
format_stamp: "Formatted at 2026-08-06 14:58:11 on dist-test-slave-68xk"
server_key: "595a14623123d4a091aa10a18a377cc3"
server_key_iv: "7202d169f76495e523c7bd2912e16487"
server_key_version: "encryptionkey@0"
I20260806 14:58:11.883433 12552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:11.920396 12552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:11.922457 12552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:11.922623 12552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:11.922866 12552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:11.923192 12552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:11.923249 12552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:11.923290 12552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:11.923316 12552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:11.930305 12552 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:37545
I20260806 14:58:11.930346 12724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:37545 every 8 connection(s)
I20260806 14:58:11.930727 12552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
Aug 06 14:58:11 dist-test-slave-68xk krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028291, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:11.938230 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 12552
I20260806 14:58:11.938354 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance
I20260806 14:58:11.938618 11297 external_mini_cluster.cc:1428] Setting key 73703e481b09fe8abb803a8ba01d56e9
I20260806 14:58:11.940531 12152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:11.931463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:45989 (local address 127.11.8.126:39037)
0806 14:58:11.931566 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:11.931569 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:11.931806 (+ 237us) server_negotiation.cc:407] Connection header received
0806 14:58:11.932857 (+ 1051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:11.932858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:11.932872 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:11.932923 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:11.933553 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:11.934062 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:11.935143 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:11.935359 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:11.935439 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:11.937469 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:11.937470 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:11.937470 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:11.937478 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:11.939071 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:11.939524 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:11.939525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:11.939526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:11.939597 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:11.939850 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:11.939851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:11.939851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:11.939966 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:11.940046 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:11.940358 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 14:58:11.940443 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 14:58:11.940699 12731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:11.931426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:39037 (local address 127.11.8.67:45989)
0806 14:58:11.931661 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:11.931711 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:11.932710 (+ 999us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:11.932993 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:11.932997 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:11.933264 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:11.933452 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:11.933456 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:11.934161 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:11.934162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:11.935048 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:11.935052 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:11.935445 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:11.935447 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:11.935577 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:11.936017 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:11.936026 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:11.937373 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:11.939197 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:11.939204 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:11.939207 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:11.939439 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:11.939673 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:11.939674 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:11.939675 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:11.939782 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:11.940037 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:11.940044 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:11.940214 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 14:58:11.940321 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 14:58:11.940471 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":87,"threads_started":1}
I20260806 14:58:11.941135 12726 heartbeater.cc:347] Connected to a master server at 127.11.8.126:39037
I20260806 14:58:11.941218 12726 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:11.941350 12726 heartbeater.cc:511] Master 127.11.8.126:39037 requested a full tablet report, sending...
I20260806 14:58:11.941689 12089 ts_manager.cc:194] Registered new tserver with Master: 0f542216756d46e991fe5321acd35942 (127.11.8.67:37545)
I20260806 14:58:11.942059 12089 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.67@KRBTEST.COM'} at 127.11.8.67:45989
I20260806 14:58:11.950291 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 14:58:11.958063 12152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:11.952311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60618 (local address 127.11.8.126:39037)
0806 14:58:11.952378 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:11.952380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:11.952491 (+ 111us) server_negotiation.cc:407] Connection header received
0806 14:58:11.952530 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:11.952531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:11.952552 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:11.952612 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:11.953125 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:11.953657 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:11.954337 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:11.954624 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:11.954690 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:11.955841 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:11.955842 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:11.955843 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:11.955849 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:11.957044 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:11.957359 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:11.957360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:11.957360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:11.957413 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:11.957572 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:11.957573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:11.957573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:11.957704 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:11.957778 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:11.957886 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 14:58:11.957947 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 14:58:11.960898 12089 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60618:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 14:58:11.962805 12089 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 14:58:11.970177 12742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:11.965928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:36947 (local address 127.0.0.1:57708)
0806 14:58:11.966120 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:11.966169 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:11.966237 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:11.966692 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:11.966696 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:11.966699 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:11.966772 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:11.966775 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:11.968088 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:11.968089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:11.969205 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:11.969207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:11.969860 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:11.969861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:11.969941 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:11.969955 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 14:58:11.970023 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 14:58:11.970086 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":28,"thread_start_us":71,"threads_started":1}
I20260806 14:58:11.970352 12747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:11.966051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57708 (local address 127.11.8.65:36947)
0806 14:58:11.966382 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:11.966388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:11.966398 (+ 10us) server_negotiation.cc:407] Connection header received
0806 14:58:11.966479 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:11.966481 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:11.966499 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:11.966582 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:11.966857 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:11.967983 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:11.969284 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:11.969770 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:11.969861 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:11.969924 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:11.970118 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 14:58:11.970166 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":101,"threads_started":1}
I20260806 14:58:11.970361 12749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:11.966152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52152 (local address 127.11.8.66:37561)
0806 14:58:11.966589 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:11.966594 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:11.966603 (+ 9us) server_negotiation.cc:407] Connection header received
0806 14:58:11.966642 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:11.966644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:11.966659 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:11.966753 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:11.966923 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:11.967733 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:11.969248 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:11.969937 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:11.970004 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:11.970053 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:11.970227 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 14:58:11.970268 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":101,"threads_started":1}
I20260806 14:58:11.970558 12748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:11.965928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:37561 (local address 127.0.0.1:52152)
0806 14:58:11.966354 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:11.966366 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:11.966464 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:11.966768 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:11.966769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:11.966771 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:11.966828 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:11.966830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:11.967840 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:11.967841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:11.969148 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:11.969151 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:11.970005 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:11.970006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:11.970072 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:11.970077 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 14:58:11.970130 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 14:58:11.970187 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":4,"spinlock_wait_cycles":768,"thread_start_us":83,"threads_started":1}
I20260806 14:58:11.970196 12746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:11.966150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:37545 (local address 127.0.0.1:50164)
0806 14:58:11.966500 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:11.966511 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:11.966569 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:11.966857 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:11.966858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:11.966860 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:11.966930 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:11.966931 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:11.968145 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:11.968146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:11.968947 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:11.968950 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:11.969972 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:11.969973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:11.970043 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:11.970046 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 14:58:11.970101 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 14:58:11.970130 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":2,"thread_start_us":78,"threads_started":1}
I20260806 14:58:11.971154 12264 tablet_service.cc:1467] Processing CreateTablet for tablet 93db877a74ce4f28baacd2e418f8a792 (DEFAULT_TABLE table=test-table [id=a21a7b8e368d400880b2432c064d2d8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:11.971257 12451 tablet_service.cc:1467] Processing CreateTablet for tablet 93db877a74ce4f28baacd2e418f8a792 (DEFAULT_TABLE table=test-table [id=a21a7b8e368d400880b2432c064d2d8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:11.971572 12264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93db877a74ce4f28baacd2e418f8a792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:11.971647 12451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93db877a74ce4f28baacd2e418f8a792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:11.974151 12754 tablet_bootstrap.cc:492] T 93db877a74ce4f28baacd2e418f8a792 P 670a0ce3d113488bbf034baef2c8418f: Bootstrap starting.
I20260806 14:58:11.970580 12750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:11.966370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50164 (local address 127.11.8.67:37545)
0806 14:58:11.966626 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:11.966632 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:11.966641 (+ 9us) server_negotiation.cc:407] Connection header received
0806 14:58:11.966679 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:11.966681 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:11.966695 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:11.966804 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:11.967016 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:11.968057 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:11.969043 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:11.969810 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:11.969883 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:11.969937 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:11.970239 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 14:58:11.970480 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":139,"threads_started":1}
I20260806 14:58:11.974838 12755 tablet_bootstrap.cc:492] T 93db877a74ce4f28baacd2e418f8a792 P 75cec4867eaa4f18adec9a71c3250282: Bootstrap starting.
I20260806 14:58:11.974926 12754 tablet_bootstrap.cc:654] T 93db877a74ce4f28baacd2e418f8a792 P 670a0ce3d113488bbf034baef2c8418f: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:11.975113 12643 tablet_service.cc:1467] Processing CreateTablet for tablet 93db877a74ce4f28baacd2e418f8a792 (DEFAULT_TABLE table=test-table [id=a21a7b8e368d400880b2432c064d2d8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:11.975472 12643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93db877a74ce4f28baacd2e418f8a792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:11.975611 12755 tablet_bootstrap.cc:654] T 93db877a74ce4f28baacd2e418f8a792 P 75cec4867eaa4f18adec9a71c3250282: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:11.975777 12754 log.cc:824] T 93db877a74ce4f28baacd2e418f8a792 P 670a0ce3d113488bbf034baef2c8418f: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:11.976173 12755 log.cc:824] T 93db877a74ce4f28baacd2e418f8a792 P 75cec4867eaa4f18adec9a71c3250282: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:11.976603 12754 tablet_bootstrap.cc:492] T 93db877a74ce4f28baacd2e418f8a792 P 670a0ce3d113488bbf034baef2c8418f: No bootstrap required, opened a new log
I20260806 14:58:11.976678 12754 ts_tablet_manager.cc:1397] T 93db877a74ce4f28baacd2e418f8a792 P 670a0ce3d113488bbf034baef2c8418f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 14:58:11.976910 12755 tablet_bootstrap.cc:492] T 93db877a74ce4f28baacd2e418f8a792 P 75cec4867eaa4f18adec9a71c3250282: No bootstrap required, opened a new log
I20260806 14:58:11.976995 12755 ts_tablet_manager.cc:1397] T 93db877a74ce4f28baacd2e418f8a792 P 75cec4867eaa4f18adec9a71c3250282: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 14:58:11.978755 12759 tablet_bootstrap.cc:492] T 93db877a74ce4f28baacd2e418f8a792 P 0f542216756d46e991fe5321acd35942: Bootstrap starting.
I20260806 14:58:11.978610 12755 raft_consensus.cc:348] T 93db877a74ce4f28baacd2e418f8a792 P 75cec4867eaa4f18adec9a71c3250282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75cec4867eaa4f18adec9a71c3250282" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36947 } } peers { permanent_uuid: "0f542216756d46e991fe5321acd35942" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37545 } } peers { permanent_uuid: "670a0ce3d113488bbf034baef2c8418f" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 37561 } }
I20260806 14:58:11.978806 12754 raft_consensus.cc:348] T 93db877a74ce4f28baacd2e418f8a792 P 670a0ce3d113488bbf034baef2c8418f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75cec4867eaa4f18adec9a71c3250282" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36947 } } peers { permanent_uuid: "0f542216756d46e991fe5321acd35942" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37545 } } peers { permanent_uuid: "670a0ce3d113488bbf034baef2c8418f" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 37561 } }
I20260806 14:58:11.978960 12755 raft_consensus.cc:374] T 93db877a74ce4f28baacd2e418f8a792 P 75cec4867eaa4f18adec9a71c3250282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:11.978986 12754 raft_consensus.cc:374] T 93db877a74ce4f28baacd2e418f8a792 P 670a0ce3d113488bbf034baef2c8418f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:11.979003 12755 raft_consensus.cc:729] T 93db877a74ce4f28baacd2e418f8a792 P 75cec4867eaa4f18adec9a71c3250282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75cec4867eaa4f18adec9a71c3250282, State: Initialized, Role: FOLLOWER
I20260806 14:58:11.979033 12754 raft_consensus.cc:729] T 93db877a74ce4f28baacd2e418f8a792 P 670a0ce3d113488bbf034baef2c8418f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 670a0ce3d113488bbf034baef2c8418f, State: Initialized, Role: FOLLOWER
I20260806 14:58:11.979159 12754 consensus_queue.cc:260] T 93db877a74ce4f28baacd2e418f8a792 P 670a0ce3d113488bbf034baef2c8418f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75cec4867eaa4f18adec9a71c3250282" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36947 } } peers { permanent_uuid: "0f542216756d46e991fe5321acd35942" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37545 } } peers { permanent_uuid: "670a0ce3d113488bbf034baef2c8418f" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 37561 } }
I20260806 14:58:11.979162 12755 consensus_queue.cc:260] T 93db877a74ce4f28baacd2e418f8a792 P 75cec4867eaa4f18adec9a71c3250282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75cec4867eaa4f18adec9a71c3250282" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36947 } } peers { permanent_uuid: "0f542216756d46e991fe5321acd35942" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37545 } } peers { permanent_uuid: "670a0ce3d113488bbf034baef2c8418f" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 37561 } }
I20260806 14:58:11.979472 12759 tablet_bootstrap.cc:654] T 93db877a74ce4f28baacd2e418f8a792 P 0f542216756d46e991fe5321acd35942: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:11.979475 12755 ts_tablet_manager.cc:1428] T 93db877a74ce4f28baacd2e418f8a792 P 75cec4867eaa4f18adec9a71c3250282: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 14:58:11.979519 12547 heartbeater.cc:503] Master 127.11.8.126:39037 was elected leader, sending a full tablet report...
I20260806 14:58:11.979590 12352 heartbeater.cc:503] Master 127.11.8.126:39037 was elected leader, sending a full tablet report...
I20260806 14:58:11.979501 12754 ts_tablet_manager.cc:1428] T 93db877a74ce4f28baacd2e418f8a792 P 670a0ce3d113488bbf034baef2c8418f: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260806 14:58:11.979980 12759 log.cc:824] T 93db877a74ce4f28baacd2e418f8a792 P 0f542216756d46e991fe5321acd35942: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:11.980731 12759 tablet_bootstrap.cc:492] T 93db877a74ce4f28baacd2e418f8a792 P 0f542216756d46e991fe5321acd35942: No bootstrap required, opened a new log
I20260806 14:58:11.980836 12759 ts_tablet_manager.cc:1397] T 93db877a74ce4f28baacd2e418f8a792 P 0f542216756d46e991fe5321acd35942: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 14:58:11.982153 12759 raft_consensus.cc:348] T 93db877a74ce4f28baacd2e418f8a792 P 0f542216756d46e991fe5321acd35942 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75cec4867eaa4f18adec9a71c3250282" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36947 } } peers { permanent_uuid: "0f542216756d46e991fe5321acd35942" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37545 } } peers { permanent_uuid: "670a0ce3d113488bbf034baef2c8418f" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 37561 } }
I20260806 14:58:11.982403 12759 raft_consensus.cc:374] T 93db877a74ce4f28baacd2e418f8a792 P 0f542216756d46e991fe5321acd35942 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:11.982537 12759 raft_consensus.cc:729] T 93db877a74ce4f28baacd2e418f8a792 P 0f542216756d46e991fe5321acd35942 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f542216756d46e991fe5321acd35942, State: Initialized, Role: FOLLOWER
I20260806 14:58:11.982688 12759 consensus_queue.cc:260] T 93db877a74ce4f28baacd2e418f8a792 P 0f542216756d46e991fe5321acd35942 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75cec4867eaa4f18adec9a71c3250282" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36947 } } peers { permanent_uuid: "0f542216756d46e991fe5321acd35942" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37545 } } peers { permanent_uuid: "670a0ce3d113488bbf034baef2c8418f" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 37561 } }
I20260806 14:58:11.982967 12759 ts_tablet_manager.cc:1428] T 93db877a74ce4f28baacd2e418f8a792 P 0f542216756d46e991fe5321acd35942: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 14:58:11.983160 12726 heartbeater.cc:503] Master 127.11.8.126:39037 was elected leader, sending a full tablet report...
W20260806 14:58:11.998620 12548 tablet.cc:2369] T 93db877a74ce4f28baacd2e418f8a792 P 670a0ce3d113488bbf034baef2c8418f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 14:58:12.095960 12353 tablet.cc:2369] T 93db877a74ce4f28baacd2e418f8a792 P 75cec4867eaa4f18adec9a71c3250282: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 14:58:12.181349 12728 tablet.cc:2369] T 93db877a74ce4f28baacd2e418f8a792 P 0f542216756d46e991fe5321acd35942: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 14:58:12.272245 12761 raft_consensus.cc:482] T 93db877a74ce4f28baacd2e418f8a792 P 75cec4867eaa4f18adec9a71c3250282 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 14:58:12.272369 12761 raft_consensus.cc:504] T 93db877a74ce4f28baacd2e418f8a792 P 75cec4867eaa4f18adec9a71c3250282 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75cec4867eaa4f18adec9a71c3250282" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36947 } } peers { permanent_uuid: "0f542216756d46e991fe5321acd35942" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37545 } } peers { permanent_uuid: "670a0ce3d113488bbf034baef2c8418f" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 37561 } }
I20260806 14:58:12.273095 12761 leader_election.cc:288] T 93db877a74ce4f28baacd2e418f8a792 P 75cec4867eaa4f18adec9a71c3250282 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f542216756d46e991fe5321acd35942 (127.11.8.67:37545), 670a0ce3d113488bbf034baef2c8418f (127.11.8.66:37561)
I20260806 14:58:12.277302 12750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:12.273499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:57997 (local address 127.11.8.67:37545)
0806 14:58:12.273549 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:12.273551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:12.273558 (+ 7us) server_negotiation.cc:407] Connection header received
0806 14:58:12.273584 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:12.273585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:12.273594 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:12.273659 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:12.273895 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:12.274669 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:12.276315 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:12.276875 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:12.276944 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:12.276955 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:12.277190 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 14:58:12.277235 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 14:58:12.277310 12749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:12.273384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:35615 (local address 127.11.8.66:37561)
0806 14:58:12.273441 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:12.273444 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:12.273508 (+ 64us) server_negotiation.cc:407] Connection header received
0806 14:58:12.273566 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:12.273568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:12.273581 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:12.273659 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:12.273892 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:12.274669 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:12.276315 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:12.276862 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:12.276944 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:12.276954 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:12.277214 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 14:58:12.277253 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 14:58:12.277348 12766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:12.273287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:37545 (local address 127.11.8.65:57997)
0806 14:58:12.273419 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:12.273431 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:12.273496 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:12.273686 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:12.273687 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:12.273691 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:12.273806 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:12.273809 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:12.274768 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:12.274768 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:12.276215 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:12.276219 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:12.276991 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:12.276992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:12.277107 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:12.277111 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 14:58:12.277169 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 14:58:12.277222 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":131,"thread_start_us":60,"threads_started":1}
I20260806 14:58:12.277359 12767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:12.273287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:37561 (local address 127.11.8.65:35615)
0806 14:58:12.273419 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:12.273431 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:12.273497 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:12.273686 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:12.273687 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:12.273689 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:12.273806 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:12.273809 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:12.274767 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:12.274768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:12.276215 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:12.276219 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:12.276988 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:12.276990 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:12.277082 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:12.277088 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 14:58:12.277165 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 14:58:12.277222 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":119,"thread_start_us":60,"threads_started":1}
I20260806 14:58:12.277827 12665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93db877a74ce4f28baacd2e418f8a792" candidate_uuid: "75cec4867eaa4f18adec9a71c3250282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f542216756d46e991fe5321acd35942" is_pre_election: true
I20260806 14:58:12.277844 12477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93db877a74ce4f28baacd2e418f8a792" candidate_uuid: "75cec4867eaa4f18adec9a71c3250282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "670a0ce3d113488bbf034baef2c8418f" is_pre_election: true
I20260806 14:58:12.278023 12665 raft_consensus.cc:2454] T 93db877a74ce4f28baacd2e418f8a792 P 0f542216756d46e991fe5321acd35942 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75cec4867eaa4f18adec9a71c3250282 in term 0.
I20260806 14:58:12.278033 12477 raft_consensus.cc:2454] T 93db877a74ce4f28baacd2e418f8a792 P 670a0ce3d113488bbf034baef2c8418f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75cec4867eaa4f18adec9a71c3250282 in term 0.
I20260806 14:58:12.278441 12202 leader_election.cc:302] T 93db877a74ce4f28baacd2e418f8a792 P 75cec4867eaa4f18adec9a71c3250282 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 670a0ce3d113488bbf034baef2c8418f, 75cec4867eaa4f18adec9a71c3250282; no voters:
I20260806 14:58:12.278584 12761 raft_consensus.cc:2781] T 93db877a74ce4f28baacd2e418f8a792 P 75cec4867eaa4f18adec9a71c3250282 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 14:58:12.278672 12761 raft_consensus.cc:482] T 93db877a74ce4f28baacd2e418f8a792 P 75cec4867eaa4f18adec9a71c3250282 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 14:58:12.278707 12761 raft_consensus.cc:3037] T 93db877a74ce4f28baacd2e418f8a792 P 75cec4867eaa4f18adec9a71c3250282 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:12.279507 12761 raft_consensus.cc:504] T 93db877a74ce4f28baacd2e418f8a792 P 75cec4867eaa4f18adec9a71c3250282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75cec4867eaa4f18adec9a71c3250282" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36947 } } peers { permanent_uuid: "0f542216756d46e991fe5321acd35942" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37545 } } peers { permanent_uuid: "670a0ce3d113488bbf034baef2c8418f" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 37561 } }
I20260806 14:58:12.279656 12761 leader_election.cc:288] T 93db877a74ce4f28baacd2e418f8a792 P 75cec4867eaa4f18adec9a71c3250282 [CANDIDATE]: Term 1 election: Requested vote from peers 0f542216756d46e991fe5321acd35942 (127.11.8.67:37545), 670a0ce3d113488bbf034baef2c8418f (127.11.8.66:37561)
I20260806 14:58:12.279860 12477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93db877a74ce4f28baacd2e418f8a792" candidate_uuid: "75cec4867eaa4f18adec9a71c3250282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "670a0ce3d113488bbf034baef2c8418f"
I20260806 14:58:12.279935 12477 raft_consensus.cc:3037] T 93db877a74ce4f28baacd2e418f8a792 P 670a0ce3d113488bbf034baef2c8418f [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:12.280529 12665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93db877a74ce4f28baacd2e418f8a792" candidate_uuid: "75cec4867eaa4f18adec9a71c3250282" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f542216756d46e991fe5321acd35942"
I20260806 14:58:12.280616 12665 raft_consensus.cc:3037] T 93db877a74ce4f28baacd2e418f8a792 P 0f542216756d46e991fe5321acd35942 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:12.280893 12477 raft_consensus.cc:2454] T 93db877a74ce4f28baacd2e418f8a792 P 670a0ce3d113488bbf034baef2c8418f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75cec4867eaa4f18adec9a71c3250282 in term 1.
I20260806 14:58:12.281085 12202 leader_election.cc:302] T 93db877a74ce4f28baacd2e418f8a792 P 75cec4867eaa4f18adec9a71c3250282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 670a0ce3d113488bbf034baef2c8418f, 75cec4867eaa4f18adec9a71c3250282; no voters:
I20260806 14:58:12.281181 12761 raft_consensus.cc:2781] T 93db877a74ce4f28baacd2e418f8a792 P 75cec4867eaa4f18adec9a71c3250282 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:12.281322 12761 raft_consensus.cc:686] T 93db877a74ce4f28baacd2e418f8a792 P 75cec4867eaa4f18adec9a71c3250282 [term 1 LEADER]: Becoming Leader. State: Replica: 75cec4867eaa4f18adec9a71c3250282, State: Running, Role: LEADER
I20260806 14:58:12.281438 12761 consensus_queue.cc:237] T 93db877a74ce4f28baacd2e418f8a792 P 75cec4867eaa4f18adec9a71c3250282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75cec4867eaa4f18adec9a71c3250282" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36947 } } peers { permanent_uuid: "0f542216756d46e991fe5321acd35942" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37545 } } peers { permanent_uuid: "670a0ce3d113488bbf034baef2c8418f" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 37561 } }
I20260806 14:58:12.281643 12665 raft_consensus.cc:2454] T 93db877a74ce4f28baacd2e418f8a792 P 0f542216756d46e991fe5321acd35942 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75cec4867eaa4f18adec9a71c3250282 in term 1.
I20260806 14:58:12.282536 12088 catalog_manager.cc:5495] T 93db877a74ce4f28baacd2e418f8a792 P 75cec4867eaa4f18adec9a71c3250282 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75cec4867eaa4f18adec9a71c3250282 (127.11.8.65). New cstate: current_term: 1 leader_uuid: "75cec4867eaa4f18adec9a71c3250282" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75cec4867eaa4f18adec9a71c3250282" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f542216756d46e991fe5321acd35942" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "670a0ce3d113488bbf034baef2c8418f" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 37561 } health_report { overall_health: UNKNOWN } } }
Aug 06 14:58:12 dist-test-slave-68xk krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.65@KRBTEST.COM
I20260806 14:58:12.333469 12747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:12.324810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57724 (local address 127.11.8.65:36947)
0806 14:58:12.324925 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:12.324928 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:12.325047 (+ 119us) server_negotiation.cc:407] Connection header received
0806 14:58:12.325127 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:12.325128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:12.325143 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:12.325192 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:12.325813 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:12.326308 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:12.327152 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:12.327410 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:12.327515 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:12.330013 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:12.330025 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:12.330025 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:12.330033 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:12.332187 (+ 2154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:12.332555 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:12.332560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:12.332560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:12.332639 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:12.332862 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:12.332863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:12.332863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:12.333033 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:12.333203 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:12.333283 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 14:58:12.333363 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
Aug 06 14:58:12 dist-test-slave-68xk krb5kdc[12028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028292, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 14:58:12 dist-test-slave-68xk krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028292, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.8.65@KRBTEST.COM
I20260806 14:58:12.357343 12747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:12.350251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57740 (local address 127.11.8.65:36947)
0806 14:58:12.350361 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:12.350363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:12.350523 (+ 160us) server_negotiation.cc:407] Connection header received
0806 14:58:12.350579 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:12.350580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:12.350589 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:12.350639 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:12.351241 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:12.351738 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:12.352489 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:12.352667 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:12.352723 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:12.354963 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:12.354964 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:12.354965 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:12.354971 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:12.356279 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:12.356599 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:12.356599 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:12.356600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:12.356648 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:12.356821 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:12.356821 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:12.356822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:12.356908 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:12.356995 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:12.357111 (+ 116us) server_negotiation.cc:299] Negotiation successful
0806 14:58:12.357197 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Aug 06 14:58:12 dist-test-slave-68xk krb5kdc[12028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 14:58:12 dist-test-slave-68xk krb5kdc[12028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.65@KRBTEST.COM
I20260806 14:58:12.380720 12747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:12.373450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57754 (local address 127.11.8.65:36947)
0806 14:58:12.373525 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:12.373527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:12.373658 (+ 131us) server_negotiation.cc:407] Connection header received
0806 14:58:12.373739 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:12.373740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:12.373757 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:12.373825 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:12.374402 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:12.374925 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:12.375666 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:12.375864 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:12.375972 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:12.378241 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:12.378243 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:12.378243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:12.378251 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:12.379541 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:12.379909 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:12.379910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:12.379910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:12.379980 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:12.380187 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:12.380187 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:12.380188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:12.380289 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:12.380412 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:12.380535 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 14:58:12.380614 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260806 14:58:12.381484 12237 tablet_service.cc:2908] Rejecting scan request for tablet 93db877a74ce4f28baacd2e418f8a792: Uninitialized: clean time has not yet been initialized
I20260806 14:58:12.382701 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 12176
I20260806 14:58:12.392858 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 12362
I20260806 14:58:12.400415 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 12552
I20260806 14:58:12.408320 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 12045
2026-08-06T14:58:12Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2432 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 14:58:12 dist-test-slave-68xk krb5kdc[12789](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 14:58:12 dist-test-slave-68xk krb5kdc[12789](info): set up 2 sockets
Aug 06 14:58:12 dist-test-slave-68xk krb5kdc[12789](info): commencing operation
krb5kdc: starting...
W20260806 14:58:13.626024 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 14:58:13 dist-test-slave-68xk krb5kdc[12789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T14:58:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T14:58:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 14:58:13.761747 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:44839
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:41261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:44839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:13.847786 12809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:13.847993 12809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:13.848016 12809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:13.849069 12809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 14:58:13.849103 12809 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 14:58:13.849124 12809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:13.849138 12809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:13.849150 12809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 14:58:13.849162 12809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 14:58:13.850569 12809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:41261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:44839
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:44839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:13.850762 12809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:13.850927 12809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:13.855775 12817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:13.855782 12816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:13.855799 12819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:13.855945 12809 server_base.cc:1034] running on GCE node
I20260806 14:58:13.856267 12809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:13.856619 12809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:13.857772 12809 hybrid_clock.cc:648] HybridClock initialized: now 1786028293857743 us; error 47 us; skew 500 ppm
Aug 06 14:58:13 dist-test-slave-68xk krb5kdc[12789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028293, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:13.860330 12809 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 14:58:13.860797 12809 webserver.cc:460] Webserver started at http://127.11.8.126:38559/ using document root <none> and password file <none>
I20260806 14:58:13.860980 12809 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:13.861020 12809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:13.861107 12809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:13.862069 12809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "76641eb070df4b3882f0487226bc0962"
format_stamp: "Formatted at 2026-08-06 14:58:13 on dist-test-slave-68xk"
server_key: "943dc76a42ef8ca78a92e1df658a2214"
server_key_iv: "a1efd34580187a7b12f0fe617526e757"
server_key_version: "encryptionkey@0"
I20260806 14:58:13.862468 12809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "76641eb070df4b3882f0487226bc0962"
format_stamp: "Formatted at 2026-08-06 14:58:13 on dist-test-slave-68xk"
server_key: "943dc76a42ef8ca78a92e1df658a2214"
server_key_iv: "a1efd34580187a7b12f0fe617526e757"
server_key_version: "encryptionkey@0"
I20260806 14:58:13.863828 12809 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 14:58:13.864599 12827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:13.864802 12809 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 14:58:13.864923 12809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "76641eb070df4b3882f0487226bc0962"
format_stamp: "Formatted at 2026-08-06 14:58:13 on dist-test-slave-68xk"
server_key: "943dc76a42ef8ca78a92e1df658a2214"
server_key_iv: "a1efd34580187a7b12f0fe617526e757"
server_key_version: "encryptionkey@0"
I20260806 14:58:13.865000 12809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:13.877734 12809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:13.879700 12809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:13.879868 12809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:13.884974 12809 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:44839
I20260806 14:58:13.885042 12910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:44839 every 8 connection(s)
I20260806 14:58:13.885373 12809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 14:58:13.886013 12913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:13.886917 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 12809
I20260806 14:58:13.887001 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 14:58:13.887199 11297 external_mini_cluster.cc:1428] Setting key be17ed4068c5a68da0b8cbf54fa0083e
I20260806 14:58:13.888676 12913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76641eb070df4b3882f0487226bc0962: Bootstrap starting.
I20260806 14:58:13.889312 12913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76641eb070df4b3882f0487226bc0962: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:13.889609 12913 log.cc:824] T 00000000000000000000000000000000 P 76641eb070df4b3882f0487226bc0962: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:13.890318 12913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76641eb070df4b3882f0487226bc0962: No bootstrap required, opened a new log
I20260806 14:58:13.892784 12913 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76641eb070df4b3882f0487226bc0962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76641eb070df4b3882f0487226bc0962" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44839 } }
I20260806 14:58:13.892967 12913 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76641eb070df4b3882f0487226bc0962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:13.893007 12913 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76641eb070df4b3882f0487226bc0962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76641eb070df4b3882f0487226bc0962, State: Initialized, Role: FOLLOWER
I20260806 14:58:13.893143 12913 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76641eb070df4b3882f0487226bc0962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76641eb070df4b3882f0487226bc0962" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44839 } }
I20260806 14:58:13.893251 12913 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76641eb070df4b3882f0487226bc0962 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 14:58:13.893287 12913 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76641eb070df4b3882f0487226bc0962 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 14:58:13.893344 12913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76641eb070df4b3882f0487226bc0962 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:13.894064 12913 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76641eb070df4b3882f0487226bc0962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76641eb070df4b3882f0487226bc0962" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44839 } }
I20260806 14:58:13.894194 12913 leader_election.cc:302] T 00000000000000000000000000000000 P 76641eb070df4b3882f0487226bc0962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76641eb070df4b3882f0487226bc0962; no voters:
I20260806 14:58:13.894402 12913 leader_election.cc:288] T 00000000000000000000000000000000 P 76641eb070df4b3882f0487226bc0962 [CANDIDATE]: Term 1 election: Requested vote from peers
Aug 06 14:58:13 dist-test-slave-68xk krb5kdc[12789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:13.894448 12921 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76641eb070df4b3882f0487226bc0962 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:13.894567 12921 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76641eb070df4b3882f0487226bc0962 [term 1 LEADER]: Becoming Leader. State: Replica: 76641eb070df4b3882f0487226bc0962, State: Running, Role: LEADER
I20260806 14:58:13.894701 12913 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76641eb070df4b3882f0487226bc0962 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 14:58:13.894719 12921 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76641eb070df4b3882f0487226bc0962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76641eb070df4b3882f0487226bc0962" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44839 } }
I20260806 14:58:13.895185 12922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76641eb070df4b3882f0487226bc0962 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76641eb070df4b3882f0487226bc0962" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76641eb070df4b3882f0487226bc0962" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44839 } } }
I20260806 14:58:13.895279 12922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76641eb070df4b3882f0487226bc0962 [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:13.895527 12923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76641eb070df4b3882f0487226bc0962 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76641eb070df4b3882f0487226bc0962. Latest consensus state: current_term: 1 leader_uuid: "76641eb070df4b3882f0487226bc0962" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76641eb070df4b3882f0487226bc0962" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44839 } } }
I20260806 14:58:13.895815 12923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76641eb070df4b3882f0487226bc0962 [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:13.896159 12929 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 14:58:13.897408 12929 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 14:58:13.898542 12917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:13.887774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52362 (local address 127.11.8.126:44839)
0806 14:58:13.887976 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:13.887984 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:13.888010 (+ 26us) server_negotiation.cc:407] Connection header received
0806 14:58:13.888477 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:13.888484 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:13.888542 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:13.888700 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:13.889353 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:13.890378 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:13.891607 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:13.892027 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:13.892158 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:13.895047 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:13.895051 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:13.895052 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:13.895063 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:13.896934 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:13.897307 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:13.897310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:13.897310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:13.897411 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:13.897625 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:13.897626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:13.897627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:13.897872 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:13.898039 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:13.898139 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 14:58:13.898292 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":90,"threads_started":1}
I20260806 14:58:13.899189 12929 catalog_manager.cc:1347] Generated new cluster ID: 391ea0f6a145458cb01c8c467b96e1f8
I20260806 14:58:13.899255 12929 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 14:58:13.907667 12929 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 14:58:13.908177 12929 catalog_manager.cc:1504] Loading token signing keys...
I20260806 14:58:13.915113 12929 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76641eb070df4b3882f0487226bc0962: Generated new TSK 0
I20260806 14:58:13.915333 12929 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
I20260806 14:58:13.949852 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:0
--local_ip_for_outbound_sockets=127.11.8.65
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:44839
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:41261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:14.056825 12949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:14.057068 12949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:14.057111 12949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:14.058296 12949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:14.058364 12949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:14.058398 12949 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 14:58:14.058455 12949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 14:58:14.059947 12949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:41261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.8.126:44839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:14.060227 12949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:14.060454 12949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:14.065781 12958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:14.065790 12955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:14.065795 12956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:14.065809 12949 server_base.cc:1034] running on GCE node
I20260806 14:58:14.066280 12949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:14.066620 12949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:14.067814 12949 hybrid_clock.cc:648] HybridClock initialized: now 1786028294067790 us; error 37 us; skew 500 ppm
Aug 06 14:58:14 dist-test-slave-68xk krb5kdc[12789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028294, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:14.070657 12949 init.cc:377] Logged in from keytab as kudu/127.11.8.65@KRBTEST.COM (short username kudu)
I20260806 14:58:14.071120 12949 webserver.cc:460] Webserver started at http://127.11.8.65:41127/ using document root <none> and password file <none>
I20260806 14:58:14.071426 12949 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:14.071507 12949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:14.071678 12949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:14.072805 12949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-0/data/instance:
uuid: "9a175ba33bac437bac96d63e54070297"
format_stamp: "Formatted at 2026-08-06 14:58:14 on dist-test-slave-68xk"
server_key: "dbeeb3ce50facc9e6279a7aad078a7a6"
server_key_iv: "1fe393488ee15806885b63986051995e"
server_key_version: "encryptionkey@0"
I20260806 14:58:14.073208 12949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance:
uuid: "9a175ba33bac437bac96d63e54070297"
format_stamp: "Formatted at 2026-08-06 14:58:14 on dist-test-slave-68xk"
server_key: "dbeeb3ce50facc9e6279a7aad078a7a6"
server_key_iv: "1fe393488ee15806885b63986051995e"
server_key_version: "encryptionkey@0"
I20260806 14:58:14.074697 12949 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 14:58:14.075497 12966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:14.075678 12949 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 14:58:14.075794 12949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "9a175ba33bac437bac96d63e54070297"
format_stamp: "Formatted at 2026-08-06 14:58:14 on dist-test-slave-68xk"
server_key: "dbeeb3ce50facc9e6279a7aad078a7a6"
server_key_iv: "1fe393488ee15806885b63986051995e"
server_key_version: "encryptionkey@0"
I20260806 14:58:14.075875 12949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:14.088464 12949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:14.091046 12949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:14.091240 12949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:14.091774 12949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:14.091840 12949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:14.091886 12949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:14.091948 12949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 14:58:14 dist-test-slave-68xk krb5kdc[12789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028294, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:14.099951 12949 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:35315
I20260806 14:58:14.100033 13110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:35315 every 8 connection(s)
I20260806 14:58:14.100426 12949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
I20260806 14:58:14.102984 12917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:14.092416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:33807 (local address 127.11.8.126:44839)
0806 14:58:14.092461 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:14.092463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:14.092728 (+ 265us) server_negotiation.cc:407] Connection header received
0806 14:58:14.093240 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:14.093242 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:14.093257 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:14.093324 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:14.094143 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.094944 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.096821 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.097176 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.097256 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:14.099943 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:14.099945 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:14.099946 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:14.099954 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:14.101522 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:14.101953 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:14.101955 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:14.101955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:14.102017 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:14.102273 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:14.102274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:14.102275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:14.102390 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:14.102506 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:14.102787 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 14:58:14.102891 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 14:58:14.103132 12979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:14.092411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:44839 (local address 127.11.8.65:33807)
0806 14:58:14.092590 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:14.092626 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:14.093123 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:14.093396 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:14.093400 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:14.093759 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:14.094045 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:14.094052 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.095063 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.095064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:14.096721 (+ 1657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:14.096726 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.097306 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.097307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:14.097483 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:14.097992 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:14.098006 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:14.099839 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:14.101627 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:14.101634 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:14.101637 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:14.101874 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:14.102086 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:14.102087 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:14.102088 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:14.102202 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:14.102500 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:14.102507 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:14.102646 (+ 139us) client_negotiation.cc:769] Sending connection context
0806 14:58:14.102752 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 14:58:14.102911 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":104,"threads_started":1}
I20260806 14:58:14.103660 13112 heartbeater.cc:347] Connected to a master server at 127.11.8.126:44839
I20260806 14:58:14.103781 13112 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:14.103976 13112 heartbeater.cc:511] Master 127.11.8.126:44839 requested a full tablet report, sending...
I20260806 14:58:14.104530 12847 ts_manager.cc:194] Registered new tserver with Master: 9a175ba33bac437bac96d63e54070297 (127.11.8.65:35315)
I20260806 14:58:14.105551 12847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.65@KRBTEST.COM'} at 127.11.8.65:33807
I20260806 14:58:14.106021 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 12949
I20260806 14:58:14.106143 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance
I20260806 14:58:14.106343 11297 external_mini_cluster.cc:1428] Setting key f1c499e47ad0e6b448538d80fa528d8c
I20260806 14:58:14.111032 13120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:14.104731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:44839 (local address 127.11.8.65:50879)
0806 14:58:14.104873 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:14.104885 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:14.104954 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:14.105085 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:14.105086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:14.105309 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:14.105366 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:14.105367 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.106135 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.106136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:14.107051 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:14.107053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.107433 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.107434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:14.107511 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:14.107971 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:14.107975 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:14.108272 (+ 297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:14.109923 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:14.109924 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:14.109924 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:14.110112 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:14.110330 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:14.110330 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:14.110330 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:14.110389 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:14.110748 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:14.110749 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:14.110792 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 14:58:14.110863 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 14:58:14.110945 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":30,"thread_start_us":81,"threads_started":1}
I20260806 14:58:14.111132 12917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:14.104809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:50879 (local address 127.11.8.126:44839)
0806 14:58:14.104854 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:14.104856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:14.104943 (+ 87us) server_negotiation.cc:407] Connection header received
0806 14:58:14.105013 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:14.105014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:14.105034 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:14.105079 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:14.105434 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.106029 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.107140 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.107345 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.107430 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:14.108363 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:14.108365 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:14.108365 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:14.108373 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:14.109853 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:14.110194 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:14.110195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:14.110196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:14.110259 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:14.110511 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:14.110512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:14.110513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:14.110653 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:14.110734 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:14.110907 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 14:58:14.110996 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
WARNING: no policy specified for kudu/127.11.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
I20260806 14:58:14.133394 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:0
--local_ip_for_outbound_sockets=127.11.8.66
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:44839
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:41261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:14.216001 13125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:14.216256 13125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:14.216298 13125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:14.217465 13125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:14.217535 13125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:14.217569 13125 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 14:58:14.217612 13125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 14:58:14.219105 13125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:41261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.8.126:44839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:14.219377 13125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:14.219610 13125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:14.224723 13133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:14.224817 13125 server_base.cc:1034] running on GCE node
W20260806 14:58:14.224748 13135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:14.224725 13132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:14.225181 13125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:14.225442 13125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:14.226585 13125 hybrid_clock.cc:648] HybridClock initialized: now 1786028294226577 us; error 30 us; skew 500 ppm
Aug 06 14:58:14 dist-test-slave-68xk krb5kdc[12789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028294, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:14.229245 13125 init.cc:377] Logged in from keytab as kudu/127.11.8.66@KRBTEST.COM (short username kudu)
I20260806 14:58:14.229686 13125 webserver.cc:460] Webserver started at http://127.11.8.66:34741/ using document root <none> and password file <none>
I20260806 14:58:14.229913 13125 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:14.229960 13125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:14.230139 13125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:14.231259 13125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-1/data/instance:
uuid: "ecd5818a9ded4cab8c333d87e5e52076"
format_stamp: "Formatted at 2026-08-06 14:58:14 on dist-test-slave-68xk"
server_key: "b4c6b804319ba18c3ab3a71e1bc81834"
server_key_iv: "9daca9c55f83bdb82a0935cf77061a7e"
server_key_version: "encryptionkey@0"
I20260806 14:58:14.231627 13125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance:
uuid: "ecd5818a9ded4cab8c333d87e5e52076"
format_stamp: "Formatted at 2026-08-06 14:58:14 on dist-test-slave-68xk"
server_key: "b4c6b804319ba18c3ab3a71e1bc81834"
server_key_iv: "9daca9c55f83bdb82a0935cf77061a7e"
server_key_version: "encryptionkey@0"
I20260806 14:58:14.233232 13125 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 14:58:14.234257 13144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:14.234508 13125 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 14:58:14.234611 13125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "ecd5818a9ded4cab8c333d87e5e52076"
format_stamp: "Formatted at 2026-08-06 14:58:14 on dist-test-slave-68xk"
server_key: "b4c6b804319ba18c3ab3a71e1bc81834"
server_key_iv: "9daca9c55f83bdb82a0935cf77061a7e"
server_key_version: "encryptionkey@0"
I20260806 14:58:14.234688 13125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:14.255718 13125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:14.258122 13125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:14.258287 13125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:14.258953 13125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:14.259004 13125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:14.259040 13125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:14.259059 13125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 14:58:14 dist-test-slave-68xk krb5kdc[12789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028294, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:14.266575 13125 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:35733
I20260806 14:58:14.266662 13304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:35733 every 8 connection(s)
I20260806 14:58:14.267030 13125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
I20260806 14:58:14.268784 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 13125
I20260806 14:58:14.268888 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance
I20260806 14:58:14.269065 11297 external_mini_cluster.cc:1428] Setting key 9eec922e1bb18ba610998d3431e2321e
I20260806 14:58:14.269065 12917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:14.259465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:37355 (local address 127.11.8.126:44839)
0806 14:58:14.259543 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:14.259545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:14.259735 (+ 190us) server_negotiation.cc:407] Connection header received
0806 14:58:14.260254 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:14.260256 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:14.260270 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:14.260354 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:14.261359 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.262235 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.263402 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.263610 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.263688 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:14.265863 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:14.265865 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:14.265866 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:14.265876 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:14.267523 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:14.268004 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:14.268005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:14.268005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:14.268070 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:14.268331 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:14.268332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:14.268333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:14.268473 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:14.268554 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:14.268891 (+ 337us) server_negotiation.cc:299] Negotiation successful
0806 14:58:14.268975 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 14:58:14.269271 13156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:14.259431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:44839 (local address 127.11.8.66:37355)
0806 14:58:14.259592 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:14.259633 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:14.260153 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:14.260475 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:14.260479 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:14.260876 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:14.261203 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:14.261210 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.262363 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.262364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:14.263304 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:14.263307 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.263725 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.263726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:14.263845 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:14.264333 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:14.264342 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:14.265761 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:14.267636 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:14.267642 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:14.267644 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:14.267871 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:14.268147 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:14.268147 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:14.268148 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:14.268261 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:14.268546 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:14.268553 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:14.268742 (+ 189us) client_negotiation.cc:769] Sending connection context
0806 14:58:14.268857 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 14:58:14.269017 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":91,"threads_started":1}
I20260806 14:58:14.269871 13305 heartbeater.cc:347] Connected to a master server at 127.11.8.126:44839
I20260806 14:58:14.269958 13305 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:14.270195 13305 heartbeater.cc:511] Master 127.11.8.126:44839 requested a full tablet report, sending...
I20260806 14:58:14.271749 12847 ts_manager.cc:194] Registered new tserver with Master: ecd5818a9ded4cab8c333d87e5e52076 (127.11.8.66:35733)
I20260806 14:58:14.272123 12847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.66@KRBTEST.COM'} at 127.11.8.66:37355
WARNING: no policy specified for kudu/127.11.8.67@KRBTEST.COM; defaulting to no policy
I20260806 14:58:14.281386 13313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:14.270911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:44839 (local address 127.11.8.66:37957)
0806 14:58:14.271042 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:14.271053 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:14.271112 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:14.272393 (+ 1281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:14.272394 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:14.272611 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:14.272687 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:14.272689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.273360 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.273361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:14.273843 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:14.273845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.274244 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.274245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:14.274322 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:14.278431 (+ 4109us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:14.278437 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:14.278821 (+ 384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:14.280404 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:14.280405 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:14.280405 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:14.280572 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:14.280749 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:14.280750 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:14.280750 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:14.280803 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:14.281133 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:14.281134 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:14.281172 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 14:58:14.281236 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 14:58:14.281319 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":81,"threads_started":1}
I20260806 14:58:14.281386 12917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:14.270978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:37957 (local address 127.11.8.126:44839)
0806 14:58:14.272258 (+ 1280us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:14.272260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:14.272269 (+ 9us) server_negotiation.cc:407] Connection header received
0806 14:58:14.272288 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:14.272289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:14.272299 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:14.272355 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:14.272792 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.273265 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.273945 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.274151 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.274242 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:14.278994 (+ 4752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:14.278995 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:14.278996 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:14.279002 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:14.280329 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:14.280639 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:14.280640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:14.280640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:14.280690 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:14.280853 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:14.280854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:14.280854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:14.281053 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:14.281133 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:14.281258 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 14:58:14.281326 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1261}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
I20260806 14:58:14.296979 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:0
--local_ip_for_outbound_sockets=127.11.8.67
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:44839
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:41261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:14.378757 13317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:14.378960 13317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:14.378984 13317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:14.380187 13317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:14.380231 13317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:14.380251 13317 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 14:58:14.380312 13317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 14:58:14.381814 13317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:41261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.8.126:44839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:14.382055 13317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:14.382278 13317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:14.387223 13323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:14.387248 13326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:14.387380 13317 server_base.cc:1034] running on GCE node
W20260806 14:58:14.387305 13324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:14.387673 13317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:14.387934 13317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:14.389078 13317 hybrid_clock.cc:648] HybridClock initialized: now 1786028294389058 us; error 38 us; skew 500 ppm
Aug 06 14:58:14 dist-test-slave-68xk krb5kdc[12789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028294, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:14.391544 13317 init.cc:377] Logged in from keytab as kudu/127.11.8.67@KRBTEST.COM (short username kudu)
I20260806 14:58:14.392076 13317 webserver.cc:460] Webserver started at http://127.11.8.67:43997/ using document root <none> and password file <none>
I20260806 14:58:14.392307 13317 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:14.392349 13317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:14.392436 13317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:14.393392 13317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-2/data/instance:
uuid: "d514f0d51b9a445e873f6f37894d92b0"
format_stamp: "Formatted at 2026-08-06 14:58:14 on dist-test-slave-68xk"
server_key: "4eecd93e6c4df06467f5aa2db805bac4"
server_key_iv: "c0aead3695b39f10894f198478e30f40"
server_key_version: "encryptionkey@0"
I20260806 14:58:14.393690 13317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance:
uuid: "d514f0d51b9a445e873f6f37894d92b0"
format_stamp: "Formatted at 2026-08-06 14:58:14 on dist-test-slave-68xk"
server_key: "4eecd93e6c4df06467f5aa2db805bac4"
server_key_iv: "c0aead3695b39f10894f198478e30f40"
server_key_version: "encryptionkey@0"
I20260806 14:58:14.395098 13317 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 14:58:14.395921 13337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:14.396128 13317 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 14:58:14.396234 13317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "d514f0d51b9a445e873f6f37894d92b0"
format_stamp: "Formatted at 2026-08-06 14:58:14 on dist-test-slave-68xk"
server_key: "4eecd93e6c4df06467f5aa2db805bac4"
server_key_iv: "c0aead3695b39f10894f198478e30f40"
server_key_version: "encryptionkey@0"
I20260806 14:58:14.396311 13317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:14.408056 13317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:14.410759 13317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:14.410926 13317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:14.411499 13317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:14.411558 13317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:14.411617 13317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:14.411648 13317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 14:58:14 dist-test-slave-68xk krb5kdc[12789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028294, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:14.419271 13317 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:36643
I20260806 14:58:14.419361 13490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:36643 every 8 connection(s)
I20260806 14:58:14.419749 13317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
I20260806 14:58:14.421984 12917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:14.411913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:57859 (local address 127.11.8.126:44839)
0806 14:58:14.411989 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:14.411992 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:14.412247 (+ 255us) server_negotiation.cc:407] Connection header received
0806 14:58:14.412998 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:14.412999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:14.413010 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:14.413088 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:14.413963 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.414582 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.415877 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.416084 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.416159 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:14.419135 (+ 2976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:14.419137 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:14.419137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:14.419146 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:14.420712 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:14.421103 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:14.421104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:14.421105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:14.421157 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:14.421448 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:14.421449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:14.421449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:14.421549 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:14.421594 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:14.421829 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 14:58:14.421902 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 14:58:14.422127 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 13317
I20260806 14:58:14.422178 13347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:14.411907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:44839 (local address 127.11.8.67:57859)
0806 14:58:14.412083 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:14.412132 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:14.412872 (+ 740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:14.413146 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:14.413151 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:14.413544 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:14.413872 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:14.413878 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.414678 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.414679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:14.415776 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:14.415779 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.416174 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.416175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:14.416314 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:14.416897 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:14.416907 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:14.419012 (+ 2105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:14.420804 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:14.420810 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:14.420811 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:14.421030 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:14.421226 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:14.421227 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:14.421228 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:14.421377 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:14.421593 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:14.421597 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:14.421721 (+ 124us) client_negotiation.cc:769] Sending connection context
0806 14:58:14.421800 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 14:58:14.421920 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":100,"threads_started":1}
I20260806 14:58:14.422226 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance
I20260806 14:58:14.422386 11297 external_mini_cluster.cc:1428] Setting key 64c6f3144667da4e4ddf8007922f90ee
I20260806 14:58:14.424026 13492 heartbeater.cc:347] Connected to a master server at 127.11.8.126:44839
I20260806 14:58:14.424111 13492 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:14.424284 13492 heartbeater.cc:511] Master 127.11.8.126:44839 requested a full tablet report, sending...
I20260806 14:58:14.424633 12846 ts_manager.cc:194] Registered new tserver with Master: d514f0d51b9a445e873f6f37894d92b0 (127.11.8.67:36643)
I20260806 14:58:14.425055 12846 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.67@KRBTEST.COM'} at 127.11.8.67:57859
I20260806 14:58:14.429526 12917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:14.424216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:57993 (local address 127.11.8.126:44839)
0806 14:58:14.424255 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:14.424258 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:14.424370 (+ 112us) server_negotiation.cc:407] Connection header received
0806 14:58:14.424442 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:14.424443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:14.424453 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:14.424498 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:14.424867 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.425400 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.426105 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.426298 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.426374 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:14.427214 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:14.427215 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:14.427216 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:14.427223 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:14.428406 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:14.428757 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:14.428758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:14.428759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:14.428818 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:14.429025 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:14.429025 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:14.429026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:14.429136 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:14.429211 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:14.429379 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 14:58:14.429445 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 14:58:14.429536 13502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:14.424119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:44839 (local address 127.11.8.67:57993)
0806 14:58:14.424292 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:14.424303 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:14.424385 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:14.424510 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:14.424511 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:14.424723 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:14.424785 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:14.424786 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.425493 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.425494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:14.426019 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:14.426020 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.426374 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.426374 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:14.426479 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:14.426861 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:14.426865 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:14.427125 (+ 260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:14.428492 (+ 1367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:14.428493 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:14.428494 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:14.428681 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:14.428886 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:14.428887 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:14.428887 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:14.428945 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:14.429204 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:14.429205 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:14.429251 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 14:58:14.429336 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 14:58:14.429442 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":71,"threads_started":1}
I20260806 14:58:14.434105 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 14:58:14 dist-test-slave-68xk krb5kdc[12789](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028294, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 14:58:14 dist-test-slave-68xk krb5kdc[12789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028294, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:14.458254 12917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:14.451219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52376 (local address 127.11.8.126:44839)
0806 14:58:14.451289 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:14.451291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:14.451299 (+ 8us) server_negotiation.cc:407] Connection header received
0806 14:58:14.451322 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:14.451323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:14.451331 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:14.451391 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:14.451957 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.452465 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.453176 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.453359 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.453425 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:14.455793 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:14.455795 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:14.455795 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:14.455801 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:14.457229 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:14.457610 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:14.457612 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:14.457612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:14.457671 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:14.457861 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:14.457861 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:14.457862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:14.457962 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:14.458043 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:14.458128 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 14:58:14.458191 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 14:58:14.459877 12846 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52376:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 14:58:14.461472 12846 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 14:58:14.468254 13521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:14.464186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:35733 (local address 127.0.0.1:58288)
0806 14:58:14.464552 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:14.464562 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:14.464603 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:14.464874 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:14.464875 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:14.464876 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:14.464914 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:14.464915 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.465763 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.465764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:14.466939 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:14.466942 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.467931 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.467932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:14.468015 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:14.468028 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 14:58:14.468104 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 14:58:14.468158 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":104,"threads_started":1}
I20260806 14:58:14.468359 13527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:14.463970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:36643 (local address 127.0.0.1:39796)
0806 14:58:14.464335 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:14.464379 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:14.464451 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:14.464766 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:14.464767 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:14.464770 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:14.464846 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:14.464848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.466229 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.466230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:14.467237 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:14.467240 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.468107 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.468109 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:14.468161 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:14.468164 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 14:58:14.468246 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 14:58:14.468284 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":9,"thread_start_us":74,"threads_started":1}
I20260806 14:58:14.468515 13528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:14.464308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58288 (local address 127.11.8.66:35733)
0806 14:58:14.464475 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:14.464479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:14.464668 (+ 189us) server_negotiation.cc:407] Connection header received
0806 14:58:14.464716 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:14.464718 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:14.464732 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:14.464807 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:14.464972 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.465670 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.467035 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.467841 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.467918 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:14.467977 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:14.468137 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 14:58:14.468188 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":79,"threads_started":1}
I20260806 14:58:14.468652 13526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:14.464159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39796 (local address 127.11.8.67:36643)
0806 14:58:14.464393 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:14.464398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:14.464542 (+ 144us) server_negotiation.cc:407] Connection header received
0806 14:58:14.464600 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:14.464602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:14.464620 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:14.464697 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:14.464934 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.466099 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.467328 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.467931 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.468004 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:14.468062 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:14.468492 (+ 430us) server_negotiation.cc:299] Negotiation successful
0806 14:58:14.468544 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":76,"threads_started":1}
I20260806 14:58:14.468719 13524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:14.464110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32770 (local address 127.11.8.65:35315)
0806 14:58:14.464568 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:14.464572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:14.464583 (+ 11us) server_negotiation.cc:407] Connection header received
0806 14:58:14.464625 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:14.464627 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:14.464644 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:14.464973 (+ 329us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:14.465055 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.466099 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.467379 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.468111 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.468180 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:14.468238 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:14.468578 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 14:58:14.468630 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":101,"threads_started":1}
I20260806 14:58:14.468501 13522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:14.463970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:35315 (local address 127.0.0.1:32770)
0806 14:58:14.464335 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:14.464379 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:14.464451 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:14.464760 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:14.464763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:14.464765 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:14.464853 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:14.464855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.466193 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.466195 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:14.467259 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:14.467262 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.468276 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.468277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:14.468340 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:14.468344 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 14:58:14.468395 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 14:58:14.468431 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":5,"thread_start_us":68,"threads_started":1}
I20260806 14:58:14.469277 13219 tablet_service.cc:1467] Processing CreateTablet for tablet 12d348edfa7c4a88a00bcff3d43827ea (DEFAULT_TABLE table=test-table [id=9102fa16e49a4442af2f95d9b9b7f686]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:14.469452 13393 tablet_service.cc:1467] Processing CreateTablet for tablet 12d348edfa7c4a88a00bcff3d43827ea (DEFAULT_TABLE table=test-table [id=9102fa16e49a4442af2f95d9b9b7f686]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:14.469550 13026 tablet_service.cc:1467] Processing CreateTablet for tablet 12d348edfa7c4a88a00bcff3d43827ea (DEFAULT_TABLE table=test-table [id=9102fa16e49a4442af2f95d9b9b7f686]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:14.469673 13219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12d348edfa7c4a88a00bcff3d43827ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:14.469794 13393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12d348edfa7c4a88a00bcff3d43827ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:14.469866 13026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12d348edfa7c4a88a00bcff3d43827ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:14.472708 13530 tablet_bootstrap.cc:492] T 12d348edfa7c4a88a00bcff3d43827ea P d514f0d51b9a445e873f6f37894d92b0: Bootstrap starting.
I20260806 14:58:14.473670 13533 tablet_bootstrap.cc:492] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076: Bootstrap starting.
I20260806 14:58:14.473745 13530 tablet_bootstrap.cc:654] T 12d348edfa7c4a88a00bcff3d43827ea P d514f0d51b9a445e873f6f37894d92b0: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:14.474083 13530 log.cc:824] T 12d348edfa7c4a88a00bcff3d43827ea P d514f0d51b9a445e873f6f37894d92b0: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:14.474359 13533 tablet_bootstrap.cc:654] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:14.474725 13533 log.cc:824] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:14.474967 13530 tablet_bootstrap.cc:492] T 12d348edfa7c4a88a00bcff3d43827ea P d514f0d51b9a445e873f6f37894d92b0: No bootstrap required, opened a new log
I20260806 14:58:14.475045 13530 ts_tablet_manager.cc:1397] T 12d348edfa7c4a88a00bcff3d43827ea P d514f0d51b9a445e873f6f37894d92b0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 14:58:14.475026 13532 tablet_bootstrap.cc:492] T 12d348edfa7c4a88a00bcff3d43827ea P 9a175ba33bac437bac96d63e54070297: Bootstrap starting.
I20260806 14:58:14.475644 13532 tablet_bootstrap.cc:654] T 12d348edfa7c4a88a00bcff3d43827ea P 9a175ba33bac437bac96d63e54070297: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:14.476025 13532 log.cc:824] T 12d348edfa7c4a88a00bcff3d43827ea P 9a175ba33bac437bac96d63e54070297: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:14.476591 13533 tablet_bootstrap.cc:492] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076: No bootstrap required, opened a new log
I20260806 14:58:14.476717 13533 ts_tablet_manager.cc:1397] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 14:58:14.476791 13532 tablet_bootstrap.cc:492] T 12d348edfa7c4a88a00bcff3d43827ea P 9a175ba33bac437bac96d63e54070297: No bootstrap required, opened a new log
I20260806 14:58:14.476866 13532 ts_tablet_manager.cc:1397] T 12d348edfa7c4a88a00bcff3d43827ea P 9a175ba33bac437bac96d63e54070297: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 14:58:14.477402 13530 raft_consensus.cc:348] T 12d348edfa7c4a88a00bcff3d43827ea P d514f0d51b9a445e873f6f37894d92b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } } peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } }
I20260806 14:58:14.477574 13530 raft_consensus.cc:374] T 12d348edfa7c4a88a00bcff3d43827ea P d514f0d51b9a445e873f6f37894d92b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:14.477613 13530 raft_consensus.cc:729] T 12d348edfa7c4a88a00bcff3d43827ea P d514f0d51b9a445e873f6f37894d92b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d514f0d51b9a445e873f6f37894d92b0, State: Initialized, Role: FOLLOWER
I20260806 14:58:14.477783 13530 consensus_queue.cc:260] T 12d348edfa7c4a88a00bcff3d43827ea P d514f0d51b9a445e873f6f37894d92b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } } peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } }
I20260806 14:58:14.478096 13492 heartbeater.cc:503] Master 127.11.8.126:44839 was elected leader, sending a full tablet report...
I20260806 14:58:14.478351 13530 ts_tablet_manager.cc:1428] T 12d348edfa7c4a88a00bcff3d43827ea P d514f0d51b9a445e873f6f37894d92b0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 14:58:14.478672 13532 raft_consensus.cc:348] T 12d348edfa7c4a88a00bcff3d43827ea P 9a175ba33bac437bac96d63e54070297 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } } peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } }
I20260806 14:58:14.478824 13532 raft_consensus.cc:374] T 12d348edfa7c4a88a00bcff3d43827ea P 9a175ba33bac437bac96d63e54070297 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:14.478888 13532 raft_consensus.cc:729] T 12d348edfa7c4a88a00bcff3d43827ea P 9a175ba33bac437bac96d63e54070297 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a175ba33bac437bac96d63e54070297, State: Initialized, Role: FOLLOWER
I20260806 14:58:14.478834 13533 raft_consensus.cc:348] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } } peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } }
I20260806 14:58:14.479029 13532 consensus_queue.cc:260] T 12d348edfa7c4a88a00bcff3d43827ea P 9a175ba33bac437bac96d63e54070297 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } } peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } }
I20260806 14:58:14.479066 13533 raft_consensus.cc:374] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:14.479121 13533 raft_consensus.cc:729] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecd5818a9ded4cab8c333d87e5e52076, State: Initialized, Role: FOLLOWER
I20260806 14:58:14.479266 13533 consensus_queue.cc:260] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } } peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } }
I20260806 14:58:14.479300 13532 ts_tablet_manager.cc:1428] T 12d348edfa7c4a88a00bcff3d43827ea P 9a175ba33bac437bac96d63e54070297: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 14:58:14.479468 13112 heartbeater.cc:503] Master 127.11.8.126:44839 was elected leader, sending a full tablet report...
I20260806 14:58:14.479606 13533 ts_tablet_manager.cc:1428] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 14:58:14.479748 13305 heartbeater.cc:503] Master 127.11.8.126:44839 was elected leader, sending a full tablet report...
W20260806 14:58:14.517700 13306 tablet.cc:2369] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 14:58:14.531584 13539 raft_consensus.cc:482] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 14:58:14.531711 13539 raft_consensus.cc:504] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } } peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } }
I20260806 14:58:14.532128 13539 leader_election.cc:288] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d514f0d51b9a445e873f6f37894d92b0 (127.11.8.67:36643), 9a175ba33bac437bac96d63e54070297 (127.11.8.65:35315)
I20260806 14:58:14.536202 13526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:14.532509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:52787 (local address 127.11.8.67:36643)
0806 14:58:14.532545 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:14.532547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:14.532651 (+ 104us) server_negotiation.cc:407] Connection header received
0806 14:58:14.532708 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:14.532709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:14.532723 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:14.532779 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:14.532933 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.533680 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.535217 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.535790 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.535870 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:14.535882 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:14.536089 (+ 207us) server_negotiation.cc:299] Negotiation successful
0806 14:58:14.536132 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260806 14:58:14.536237 13524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:14.532434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:36619 (local address 127.11.8.65:35315)
0806 14:58:14.532481 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:14.532484 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:14.532495 (+ 11us) server_negotiation.cc:407] Connection header received
0806 14:58:14.532586 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:14.532587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:14.532598 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:14.532682 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:14.532880 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.533672 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.535217 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.535794 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.535870 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:14.535881 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:14.536090 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 14:58:14.536132 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 14:58:14.536231 13540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:14.532369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:36643 (local address 127.11.8.66:52787)
0806 14:58:14.532574 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:14.532583 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:14.532634 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:14.532799 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:14.532800 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:14.532802 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:14.532861 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:14.532862 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.533760 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.533761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:14.535111 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:14.535115 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.535882 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.535884 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:14.535997 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:14.536001 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 14:58:14.536067 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 14:58:14.536116 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":83,"thread_start_us":118,"threads_started":1}
I20260806 14:58:14.536239 13156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:14.532331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:35315 (local address 127.11.8.66:36619)
0806 14:58:14.532440 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:14.532453 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:14.532503 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:14.532698 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:14.532700 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:14.532702 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:14.532797 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:14.532799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.533760 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.533761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:14.535111 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:14.535115 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.535882 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.535884 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:14.535998 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:14.536001 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 14:58:14.536067 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 14:58:14.536116 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":79}
I20260806 14:58:14.536686 13049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12d348edfa7c4a88a00bcff3d43827ea" candidate_uuid: "ecd5818a9ded4cab8c333d87e5e52076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a175ba33bac437bac96d63e54070297" is_pre_election: true
I20260806 14:58:14.536702 13436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12d348edfa7c4a88a00bcff3d43827ea" candidate_uuid: "ecd5818a9ded4cab8c333d87e5e52076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d514f0d51b9a445e873f6f37894d92b0" is_pre_election: true
I20260806 14:58:14.536854 13049 raft_consensus.cc:2454] T 12d348edfa7c4a88a00bcff3d43827ea P 9a175ba33bac437bac96d63e54070297 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ecd5818a9ded4cab8c333d87e5e52076 in term 0.
I20260806 14:58:14.536877 13436 raft_consensus.cc:2454] T 12d348edfa7c4a88a00bcff3d43827ea P d514f0d51b9a445e873f6f37894d92b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ecd5818a9ded4cab8c333d87e5e52076 in term 0.
I20260806 14:58:14.537109 13150 leader_election.cc:302] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d514f0d51b9a445e873f6f37894d92b0, ecd5818a9ded4cab8c333d87e5e52076; no voters:
I20260806 14:58:14.537246 13539 raft_consensus.cc:2781] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 14:58:14.537300 13539 raft_consensus.cc:482] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 14:58:14.537326 13539 raft_consensus.cc:3037] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:14.538556 13539 raft_consensus.cc:504] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } } peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } }
I20260806 14:58:14.538705 13539 leader_election.cc:288] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076 [CANDIDATE]: Term 1 election: Requested vote from peers d514f0d51b9a445e873f6f37894d92b0 (127.11.8.67:36643), 9a175ba33bac437bac96d63e54070297 (127.11.8.65:35315)
I20260806 14:58:14.538842 13436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12d348edfa7c4a88a00bcff3d43827ea" candidate_uuid: "ecd5818a9ded4cab8c333d87e5e52076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d514f0d51b9a445e873f6f37894d92b0"
I20260806 14:58:14.538865 13049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12d348edfa7c4a88a00bcff3d43827ea" candidate_uuid: "ecd5818a9ded4cab8c333d87e5e52076" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a175ba33bac437bac96d63e54070297"
I20260806 14:58:14.538923 13436 raft_consensus.cc:3037] T 12d348edfa7c4a88a00bcff3d43827ea P d514f0d51b9a445e873f6f37894d92b0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:14.538934 13049 raft_consensus.cc:3037] T 12d348edfa7c4a88a00bcff3d43827ea P 9a175ba33bac437bac96d63e54070297 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:14.539848 13049 raft_consensus.cc:2454] T 12d348edfa7c4a88a00bcff3d43827ea P 9a175ba33bac437bac96d63e54070297 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecd5818a9ded4cab8c333d87e5e52076 in term 1.
I20260806 14:58:14.539848 13436 raft_consensus.cc:2454] T 12d348edfa7c4a88a00bcff3d43827ea P d514f0d51b9a445e873f6f37894d92b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ecd5818a9ded4cab8c333d87e5e52076 in term 1.
I20260806 14:58:14.540047 13147 leader_election.cc:302] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a175ba33bac437bac96d63e54070297, ecd5818a9ded4cab8c333d87e5e52076; no voters:
I20260806 14:58:14.540149 13539 raft_consensus.cc:2781] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:14.540318 13539 raft_consensus.cc:686] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076 [term 1 LEADER]: Becoming Leader. State: Replica: ecd5818a9ded4cab8c333d87e5e52076, State: Running, Role: LEADER
I20260806 14:58:14.540401 13539 consensus_queue.cc:237] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } } peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } }
I20260806 14:58:14.541304 12846 catalog_manager.cc:5495] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076 reported cstate change: term changed from 0 to 1, leader changed from <none> to ecd5818a9ded4cab8c333d87e5e52076 (127.11.8.66). New cstate: current_term: 1 leader_uuid: "ecd5818a9ded4cab8c333d87e5e52076" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } health_report { overall_health: HEALTHY } } }
I20260806 14:58:14.561959 13528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:14.559449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58298 (local address 127.11.8.66:35733)
0806 14:58:14.559526 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:14.559528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:14.559538 (+ 10us) server_negotiation.cc:407] Connection header received
0806 14:58:14.559606 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:14.559613 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:14.559634 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:14.559720 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 14:58:14.559861 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.560320 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.561103 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.561304 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.561384 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:14.561593 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 14:58:14.561694 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 14:58:14.561794 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:14.561845 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 14:58:14.561868 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 14:58:14.563970 13049 raft_consensus.cc:1261] T 12d348edfa7c4a88a00bcff3d43827ea P 9a175ba33bac437bac96d63e54070297 [term 1 FOLLOWER]: Refusing update from remote peer ecd5818a9ded4cab8c333d87e5e52076: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:14.563969 13436 raft_consensus.cc:1261] T 12d348edfa7c4a88a00bcff3d43827ea P d514f0d51b9a445e873f6f37894d92b0 [term 1 FOLLOWER]: Refusing update from remote peer ecd5818a9ded4cab8c333d87e5e52076: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:14.564370 13541 consensus_queue.cc:1035] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:14.565320 13541 consensus_queue.cc:1035] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:14.567492 13546 mvcc.cc:205] Tried to move back new op lower bound from 7315571894530445312 to 7315571894438055936. Current Snapshot: MvccSnapshot[applied={T|T < 7315571894530445312}]
I20260806 14:58:14.568068 13548 mvcc.cc:205] Tried to move back new op lower bound from 7315571894530445312 to 7315571894438055936. Current Snapshot: MvccSnapshot[applied={T|T < 7315571894530445312}]
I20260806 14:58:14.571267 12846 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:58:14.571404 12846 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:58:14.572337 12846 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 76641eb070df4b3882f0487226bc0962: Sending DeleteTablet for 3 replicas of tablet 12d348edfa7c4a88a00bcff3d43827ea
I20260806 14:58:14.572860 13026 tablet_service.cc:1514] Processing DeleteTablet for tablet 12d348edfa7c4a88a00bcff3d43827ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:14 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:32770
I20260806 14:58:14.573103 13561 tablet_replica.cc:330] T 12d348edfa7c4a88a00bcff3d43827ea P 9a175ba33bac437bac96d63e54070297: stopping tablet replica
I20260806 14:58:14.573158 13219 tablet_service.cc:1514] Processing DeleteTablet for tablet 12d348edfa7c4a88a00bcff3d43827ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:14 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:58288
I20260806 14:58:14.573266 13561 raft_consensus.cc:2229] T 12d348edfa7c4a88a00bcff3d43827ea P 9a175ba33bac437bac96d63e54070297 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:58:14.573438 13562 tablet_replica.cc:330] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076: stopping tablet replica
I20260806 14:58:14.573596 13562 raft_consensus.cc:2229] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076 [term 1 LEADER]: Raft consensus shutting down.
I20260806 14:58:14.573649 13561 raft_consensus.cc:2258] T 12d348edfa7c4a88a00bcff3d43827ea P 9a175ba33bac437bac96d63e54070297 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:14.573756 13562 raft_consensus.cc:2258] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:14.574299 13562 ts_tablet_manager.cc:1905] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:14.574810 13393 tablet_service.cc:1514] Processing DeleteTablet for tablet 12d348edfa7c4a88a00bcff3d43827ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:14 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:39796
I20260806 14:58:14.575263 13561 ts_tablet_manager.cc:1905] T 12d348edfa7c4a88a00bcff3d43827ea P 9a175ba33bac437bac96d63e54070297: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:14.575393 12846 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52376:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 14:58:14.575548 12846 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 14:58:14.575907 13562 ts_tablet_manager.cc:1918] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 14:58:14.576215 13562 log.cc:1196] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-1/wal/wals/12d348edfa7c4a88a00bcff3d43827ea
I20260806 14:58:14.576574 13562 ts_tablet_manager.cc:1939] T 12d348edfa7c4a88a00bcff3d43827ea P ecd5818a9ded4cab8c333d87e5e52076: Deleting consensus metadata
I20260806 14:58:14.576987 13561 ts_tablet_manager.cc:1918] T 12d348edfa7c4a88a00bcff3d43827ea P 9a175ba33bac437bac96d63e54070297: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 14:58:14.577103 13561 log.cc:1196] T 12d348edfa7c4a88a00bcff3d43827ea P 9a175ba33bac437bac96d63e54070297: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-0/wal/wals/12d348edfa7c4a88a00bcff3d43827ea
I20260806 14:58:14.577121 12831 catalog_manager.cc:4841] TS ecd5818a9ded4cab8c333d87e5e52076 (127.11.8.66:35733): tablet 12d348edfa7c4a88a00bcff3d43827ea (table test-table [id=9102fa16e49a4442af2f95d9b9b7f686]) successfully deleted
I20260806 14:58:14.577246 13026 tablet_service.cc:1467] Processing CreateTablet for tablet 39a0ce0486754517b54af9f0f2614d0e (DEFAULT_TABLE table=test-table [id=e6ad0e5b5286475b98bc8139aa9fe7d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:14.577364 13026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39a0ce0486754517b54af9f0f2614d0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:14.577440 13561 ts_tablet_manager.cc:1939] T 12d348edfa7c4a88a00bcff3d43827ea P 9a175ba33bac437bac96d63e54070297: Deleting consensus metadata
I20260806 14:58:14.577484 13219 tablet_service.cc:1467] Processing CreateTablet for tablet 39a0ce0486754517b54af9f0f2614d0e (DEFAULT_TABLE table=test-table [id=e6ad0e5b5286475b98bc8139aa9fe7d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:14.577587 13219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39a0ce0486754517b54af9f0f2614d0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:14.578550 13532 tablet_bootstrap.cc:492] T 39a0ce0486754517b54af9f0f2614d0e P 9a175ba33bac437bac96d63e54070297: Bootstrap starting.
I20260806 14:58:14.578759 12829 catalog_manager.cc:4841] TS 9a175ba33bac437bac96d63e54070297 (127.11.8.65:35315): tablet 12d348edfa7c4a88a00bcff3d43827ea (table test-table [id=9102fa16e49a4442af2f95d9b9b7f686]) successfully deleted
I20260806 14:58:14.578851 13533 tablet_bootstrap.cc:492] T 39a0ce0486754517b54af9f0f2614d0e P ecd5818a9ded4cab8c333d87e5e52076: Bootstrap starting.
I20260806 14:58:14.578974 13532 tablet_bootstrap.cc:654] T 39a0ce0486754517b54af9f0f2614d0e P 9a175ba33bac437bac96d63e54070297: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:14.579264 13533 tablet_bootstrap.cc:654] T 39a0ce0486754517b54af9f0f2614d0e P ecd5818a9ded4cab8c333d87e5e52076: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:14.579834 13532 tablet_bootstrap.cc:492] T 39a0ce0486754517b54af9f0f2614d0e P 9a175ba33bac437bac96d63e54070297: No bootstrap required, opened a new log
I20260806 14:58:14.579885 13532 ts_tablet_manager.cc:1397] T 39a0ce0486754517b54af9f0f2614d0e P 9a175ba33bac437bac96d63e54070297: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:14.579895 13393 tablet_service.cc:1467] Processing CreateTablet for tablet 39a0ce0486754517b54af9f0f2614d0e (DEFAULT_TABLE table=test-table [id=e6ad0e5b5286475b98bc8139aa9fe7d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:14.580006 13393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39a0ce0486754517b54af9f0f2614d0e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:14.580030 13532 raft_consensus.cc:348] T 39a0ce0486754517b54af9f0f2614d0e P 9a175ba33bac437bac96d63e54070297 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } }
I20260806 14:58:14.580090 13532 raft_consensus.cc:374] T 39a0ce0486754517b54af9f0f2614d0e P 9a175ba33bac437bac96d63e54070297 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:14.580112 13532 raft_consensus.cc:729] T 39a0ce0486754517b54af9f0f2614d0e P 9a175ba33bac437bac96d63e54070297 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a175ba33bac437bac96d63e54070297, State: Initialized, Role: FOLLOWER
I20260806 14:58:14.580169 13532 consensus_queue.cc:260] T 39a0ce0486754517b54af9f0f2614d0e P 9a175ba33bac437bac96d63e54070297 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } }
I20260806 14:58:14.580265 13532 ts_tablet_manager.cc:1428] T 39a0ce0486754517b54af9f0f2614d0e P 9a175ba33bac437bac96d63e54070297: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:14.581591 13530 tablet_bootstrap.cc:492] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0: Bootstrap starting.
I20260806 14:58:14.581914 13533 tablet_bootstrap.cc:492] T 39a0ce0486754517b54af9f0f2614d0e P ecd5818a9ded4cab8c333d87e5e52076: No bootstrap required, opened a new log
I20260806 14:58:14.581962 13533 ts_tablet_manager.cc:1397] T 39a0ce0486754517b54af9f0f2614d0e P ecd5818a9ded4cab8c333d87e5e52076: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 14:58:14.582010 13530 tablet_bootstrap.cc:654] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:14.582082 13533 raft_consensus.cc:348] T 39a0ce0486754517b54af9f0f2614d0e P ecd5818a9ded4cab8c333d87e5e52076 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } }
I20260806 14:58:14.582123 13533 raft_consensus.cc:374] T 39a0ce0486754517b54af9f0f2614d0e P ecd5818a9ded4cab8c333d87e5e52076 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:14.582136 13533 raft_consensus.cc:729] T 39a0ce0486754517b54af9f0f2614d0e P ecd5818a9ded4cab8c333d87e5e52076 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecd5818a9ded4cab8c333d87e5e52076, State: Initialized, Role: FOLLOWER
I20260806 14:58:14.582195 13533 consensus_queue.cc:260] T 39a0ce0486754517b54af9f0f2614d0e P ecd5818a9ded4cab8c333d87e5e52076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } }
I20260806 14:58:14.582304 13533 ts_tablet_manager.cc:1428] T 39a0ce0486754517b54af9f0f2614d0e P ecd5818a9ded4cab8c333d87e5e52076: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:14.582671 13564 tablet_replica.cc:330] T 12d348edfa7c4a88a00bcff3d43827ea P d514f0d51b9a445e873f6f37894d92b0: stopping tablet replica
I20260806 14:58:14.582763 13530 tablet_bootstrap.cc:492] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0: No bootstrap required, opened a new log
I20260806 14:58:14.582881 13530 ts_tablet_manager.cc:1397] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:14.583071 13530 raft_consensus.cc:348] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } }
I20260806 14:58:14.583168 13530 raft_consensus.cc:374] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:14.583189 13530 raft_consensus.cc:729] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d514f0d51b9a445e873f6f37894d92b0, State: Initialized, Role: FOLLOWER
I20260806 14:58:14.583257 13530 consensus_queue.cc:260] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } }
I20260806 14:58:14.583349 13530 ts_tablet_manager.cc:1428] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:14.582885 13564 raft_consensus.cc:2229] T 12d348edfa7c4a88a00bcff3d43827ea P d514f0d51b9a445e873f6f37894d92b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:58:14.583510 13564 raft_consensus.cc:2258] T 12d348edfa7c4a88a00bcff3d43827ea P d514f0d51b9a445e873f6f37894d92b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:14.584244 13564 ts_tablet_manager.cc:1905] T 12d348edfa7c4a88a00bcff3d43827ea P d514f0d51b9a445e873f6f37894d92b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:14.585675 13564 ts_tablet_manager.cc:1918] T 12d348edfa7c4a88a00bcff3d43827ea P d514f0d51b9a445e873f6f37894d92b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 14:58:14.585755 13564 log.cc:1196] T 12d348edfa7c4a88a00bcff3d43827ea P d514f0d51b9a445e873f6f37894d92b0: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-2/wal/wals/12d348edfa7c4a88a00bcff3d43827ea
I20260806 14:58:14.585999 13564 ts_tablet_manager.cc:1939] T 12d348edfa7c4a88a00bcff3d43827ea P d514f0d51b9a445e873f6f37894d92b0: Deleting consensus metadata
I20260806 14:58:14.586382 12831 catalog_manager.cc:4841] TS d514f0d51b9a445e873f6f37894d92b0 (127.11.8.67:36643): tablet 12d348edfa7c4a88a00bcff3d43827ea (table test-table [id=9102fa16e49a4442af2f95d9b9b7f686]) successfully deleted
W20260806 14:58:14.601215 13113 tablet.cc:2369] T 39a0ce0486754517b54af9f0f2614d0e P 9a175ba33bac437bac96d63e54070297: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 14:58:14.670583 13494 tablet.cc:2369] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 14:58:14.952406 13537 raft_consensus.cc:482] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 14:58:14.952510 13537 raft_consensus.cc:504] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } }
I20260806 14:58:14.952857 13537 leader_election.cc:288] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a175ba33bac437bac96d63e54070297 (127.11.8.65:35315), ecd5818a9ded4cab8c333d87e5e52076 (127.11.8.66:35733)
I20260806 14:58:14.957127 13524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:14.953159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:42429 (local address 127.11.8.65:35315)
0806 14:58:14.953218 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:14.953221 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:14.953390 (+ 169us) server_negotiation.cc:407] Connection header received
0806 14:58:14.953446 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:14.953447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:14.953458 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:14.953525 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:14.953760 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.954532 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.956184 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.956730 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.956807 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:14.956818 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:14.957017 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 14:58:14.957054 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 14:58:14.957127 13528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:14.953202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:46741 (local address 127.11.8.66:35733)
0806 14:58:14.953252 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:14.953255 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:14.953383 (+ 128us) server_negotiation.cc:407] Connection header received
0806 14:58:14.953446 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:14.953447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:14.953458 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:14.953525 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:14.953756 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.954539 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.956184 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.956741 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.956807 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:14.956818 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:14.957017 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 14:58:14.957055 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 14:58:14.957170 13573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:14.953124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:35733 (local address 127.11.8.67:46741)
0806 14:58:14.953278 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:14.953292 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:14.953382 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:14.953550 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:14.953551 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:14.953554 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:14.953665 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:14.953666 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.954635 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.954635 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:14.956078 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:14.956082 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.956829 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.956830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:14.956919 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:14.956924 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 14:58:14.956994 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 14:58:14.957051 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":85,"thread_start_us":73,"threads_started":1}
I20260806 14:58:14.957300 13576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:14.953137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:35315 (local address 127.11.8.67:42429)
0806 14:58:14.953287 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:14.953298 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:14.953382 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:14.953550 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:14.953551 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:14.953555 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:14.953662 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:14.953665 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.954622 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.954623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:14.956078 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:14.956081 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:14.956830 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:14.956831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:14.956920 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:14.956926 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 14:58:14.956994 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 14:58:14.957051 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":57,"thread_start_us":69,"threads_started":1}
I20260806 14:58:14.957520 13244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39a0ce0486754517b54af9f0f2614d0e" candidate_uuid: "d514f0d51b9a445e873f6f37894d92b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecd5818a9ded4cab8c333d87e5e52076" is_pre_election: true
I20260806 14:58:14.957645 13049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39a0ce0486754517b54af9f0f2614d0e" candidate_uuid: "d514f0d51b9a445e873f6f37894d92b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a175ba33bac437bac96d63e54070297" is_pre_election: true
I20260806 14:58:14.957710 13244 raft_consensus.cc:2454] T 39a0ce0486754517b54af9f0f2614d0e P ecd5818a9ded4cab8c333d87e5e52076 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d514f0d51b9a445e873f6f37894d92b0 in term 0.
I20260806 14:58:14.957762 13049 raft_consensus.cc:2454] T 39a0ce0486754517b54af9f0f2614d0e P 9a175ba33bac437bac96d63e54070297 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d514f0d51b9a445e873f6f37894d92b0 in term 0.
I20260806 14:58:14.957945 13339 leader_election.cc:302] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a175ba33bac437bac96d63e54070297, d514f0d51b9a445e873f6f37894d92b0; no voters:
I20260806 14:58:14.958175 13537 raft_consensus.cc:2781] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 14:58:14.958254 13537 raft_consensus.cc:482] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 14:58:14.958288 13537 raft_consensus.cc:3037] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:14.959090 13537 raft_consensus.cc:504] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } }
I20260806 14:58:14.959249 13537 leader_election.cc:288] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0 [CANDIDATE]: Term 1 election: Requested vote from peers 9a175ba33bac437bac96d63e54070297 (127.11.8.65:35315), ecd5818a9ded4cab8c333d87e5e52076 (127.11.8.66:35733)
I20260806 14:58:14.959407 13049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39a0ce0486754517b54af9f0f2614d0e" candidate_uuid: "d514f0d51b9a445e873f6f37894d92b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a175ba33bac437bac96d63e54070297"
I20260806 14:58:14.959483 13049 raft_consensus.cc:3037] T 39a0ce0486754517b54af9f0f2614d0e P 9a175ba33bac437bac96d63e54070297 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:14.960109 13049 raft_consensus.cc:2454] T 39a0ce0486754517b54af9f0f2614d0e P 9a175ba33bac437bac96d63e54070297 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d514f0d51b9a445e873f6f37894d92b0 in term 1.
I20260806 14:58:14.960234 13244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39a0ce0486754517b54af9f0f2614d0e" candidate_uuid: "d514f0d51b9a445e873f6f37894d92b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecd5818a9ded4cab8c333d87e5e52076"
I20260806 14:58:14.960274 13339 leader_election.cc:302] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a175ba33bac437bac96d63e54070297, d514f0d51b9a445e873f6f37894d92b0; no voters:
I20260806 14:58:14.960311 13244 raft_consensus.cc:3037] T 39a0ce0486754517b54af9f0f2614d0e P ecd5818a9ded4cab8c333d87e5e52076 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:14.960376 13537 raft_consensus.cc:2781] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:14.960493 13537 raft_consensus.cc:686] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0 [term 1 LEADER]: Becoming Leader. State: Replica: d514f0d51b9a445e873f6f37894d92b0, State: Running, Role: LEADER
I20260806 14:58:14.960562 13537 consensus_queue.cc:237] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } }
I20260806 14:58:14.961225 13244 raft_consensus.cc:2454] T 39a0ce0486754517b54af9f0f2614d0e P ecd5818a9ded4cab8c333d87e5e52076 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d514f0d51b9a445e873f6f37894d92b0 in term 1.
I20260806 14:58:14.961354 12846 catalog_manager.cc:5495] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d514f0d51b9a445e873f6f37894d92b0 (127.11.8.67). New cstate: current_term: 1 leader_uuid: "d514f0d51b9a445e873f6f37894d92b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } health_report { overall_health: HEALTHY } } }
Aug 06 14:58:15 dist-test-slave-68xk krb5kdc[12789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028293, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:15.043052 13592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:15.035348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52392 (local address 127.11.8.126:44839)
0806 14:58:15.035483 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:15.035485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:15.035599 (+ 114us) server_negotiation.cc:407] Connection header received
0806 14:58:15.035675 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:15.035675 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:15.035688 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:15.035768 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:15.036450 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:15.037054 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:15.037808 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:15.037995 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:15.038082 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:15.040356 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:15.040358 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:15.040358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:15.040365 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:15.041721 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:15.042127 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:15.042128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:15.042129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:15.042190 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:15.042398 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:15.042399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:15.042399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:15.042543 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:15.042620 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:15.042794 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 14:58:15.042927 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
I20260806 14:58:15.043124 13591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:15.035258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:44839 (local address 127.0.0.1:52392)
0806 14:58:15.035492 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:15.035504 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:15.035620 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:15.035784 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:15.035786 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:15.036302 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:15.036356 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:15.036360 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:15.037146 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:15.037148 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:15.037717 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:15.037720 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:15.038086 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:15.038088 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:15.038142 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:15.038740 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:15.038744 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:15.040256 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:15.041807 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:15.041815 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:15.041818 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:15.042027 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:15.042260 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:15.042260 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:15.042261 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:15.042322 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:15.042623 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:15.042632 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:15.042693 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 14:58:15.042768 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 14:58:15.042993 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":119,"thread_start_us":126,"threads_started":1}
I20260806 14:58:15.045593 12846 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:52392:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 14:58:15.045877 12846 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 14:58:15.047683 13219 tablet_service.cc:1467] Processing CreateTablet for tablet 49cdaf4adbef4963abd02c08ee912b2e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2abc2df9a2b6449fb725d9e7402bd540]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 14:58:15.047720 13026 tablet_service.cc:1467] Processing CreateTablet for tablet 49cdaf4adbef4963abd02c08ee912b2e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2abc2df9a2b6449fb725d9e7402bd540]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 14:58:15.047784 13393 tablet_service.cc:1467] Processing CreateTablet for tablet 49cdaf4adbef4963abd02c08ee912b2e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2abc2df9a2b6449fb725d9e7402bd540]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 14:58:15.047925 13026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49cdaf4adbef4963abd02c08ee912b2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:15.047924 13393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49cdaf4adbef4963abd02c08ee912b2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:15.047927 13219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49cdaf4adbef4963abd02c08ee912b2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:15.050458 13533 tablet_bootstrap.cc:492] T 49cdaf4adbef4963abd02c08ee912b2e P ecd5818a9ded4cab8c333d87e5e52076: Bootstrap starting.
I20260806 14:58:15.050516 13532 tablet_bootstrap.cc:492] T 49cdaf4adbef4963abd02c08ee912b2e P 9a175ba33bac437bac96d63e54070297: Bootstrap starting.
I20260806 14:58:15.050516 13530 tablet_bootstrap.cc:492] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0: Bootstrap starting.
I20260806 14:58:15.050882 13533 tablet_bootstrap.cc:654] T 49cdaf4adbef4963abd02c08ee912b2e P ecd5818a9ded4cab8c333d87e5e52076: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:15.050966 13530 tablet_bootstrap.cc:654] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:15.050966 13532 tablet_bootstrap.cc:654] T 49cdaf4adbef4963abd02c08ee912b2e P 9a175ba33bac437bac96d63e54070297: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:15.051488 13533 tablet_bootstrap.cc:492] T 49cdaf4adbef4963abd02c08ee912b2e P ecd5818a9ded4cab8c333d87e5e52076: No bootstrap required, opened a new log
I20260806 14:58:15.051543 13533 ts_tablet_manager.cc:1397] T 49cdaf4adbef4963abd02c08ee912b2e P ecd5818a9ded4cab8c333d87e5e52076: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:15.051687 13533 raft_consensus.cc:348] T 49cdaf4adbef4963abd02c08ee912b2e P ecd5818a9ded4cab8c333d87e5e52076 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } }
I20260806 14:58:15.051769 13533 raft_consensus.cc:374] T 49cdaf4adbef4963abd02c08ee912b2e P ecd5818a9ded4cab8c333d87e5e52076 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:15.051791 13533 raft_consensus.cc:729] T 49cdaf4adbef4963abd02c08ee912b2e P ecd5818a9ded4cab8c333d87e5e52076 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecd5818a9ded4cab8c333d87e5e52076, State: Initialized, Role: FOLLOWER
I20260806 14:58:15.051798 13532 tablet_bootstrap.cc:492] T 49cdaf4adbef4963abd02c08ee912b2e P 9a175ba33bac437bac96d63e54070297: No bootstrap required, opened a new log
I20260806 14:58:15.051846 13532 ts_tablet_manager.cc:1397] T 49cdaf4adbef4963abd02c08ee912b2e P 9a175ba33bac437bac96d63e54070297: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:15.051836 13533 consensus_queue.cc:260] T 49cdaf4adbef4963abd02c08ee912b2e P ecd5818a9ded4cab8c333d87e5e52076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } }
I20260806 14:58:15.051947 13533 ts_tablet_manager.cc:1428] T 49cdaf4adbef4963abd02c08ee912b2e P ecd5818a9ded4cab8c333d87e5e52076: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:15.051990 13530 tablet_bootstrap.cc:492] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0: No bootstrap required, opened a new log
I20260806 14:58:15.052038 13530 ts_tablet_manager.cc:1397] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 14:58:15.051991 13532 raft_consensus.cc:348] T 49cdaf4adbef4963abd02c08ee912b2e P 9a175ba33bac437bac96d63e54070297 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } }
I20260806 14:58:15.051994 13541 tablet_replica.cc:439] T 49cdaf4adbef4963abd02c08ee912b2e P ecd5818a9ded4cab8c333d87e5e52076: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } } }
I20260806 14:58:15.052074 13532 raft_consensus.cc:374] T 49cdaf4adbef4963abd02c08ee912b2e P 9a175ba33bac437bac96d63e54070297 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:15.052096 13532 raft_consensus.cc:729] T 49cdaf4adbef4963abd02c08ee912b2e P 9a175ba33bac437bac96d63e54070297 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a175ba33bac437bac96d63e54070297, State: Initialized, Role: FOLLOWER
I20260806 14:58:15.052091 13541 tablet_replica.cc:442] T 49cdaf4adbef4963abd02c08ee912b2e P ecd5818a9ded4cab8c333d87e5e52076: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 14:58:15.052136 13532 consensus_queue.cc:260] T 49cdaf4adbef4963abd02c08ee912b2e P 9a175ba33bac437bac96d63e54070297 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } }
I20260806 14:58:15.052225 13532 ts_tablet_manager.cc:1428] T 49cdaf4adbef4963abd02c08ee912b2e P 9a175ba33bac437bac96d63e54070297: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 14:58:15.052205 13530 raft_consensus.cc:348] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } }
I20260806 14:58:15.052269 13530 raft_consensus.cc:374] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:15.052289 13530 raft_consensus.cc:729] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d514f0d51b9a445e873f6f37894d92b0, State: Initialized, Role: FOLLOWER
I20260806 14:58:15.052373 13530 consensus_queue.cc:260] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } }
I20260806 14:58:15.052345 13538 tablet_replica.cc:439] T 49cdaf4adbef4963abd02c08ee912b2e P 9a175ba33bac437bac96d63e54070297: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } } }
I20260806 14:58:15.052443 13530 ts_tablet_manager.cc:1428] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:15.052466 13538 tablet_replica.cc:442] T 49cdaf4adbef4963abd02c08ee912b2e P 9a175ba33bac437bac96d63e54070297: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 14:58:15.052502 13537 tablet_replica.cc:439] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } } }
I20260806 14:58:15.052598 13537 tablet_replica.cc:442] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 14:58:15.066532 13537 raft_consensus.cc:482] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 14:58:15.066733 13537 raft_consensus.cc:504] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } }
I20260806 14:58:15.066918 13537 leader_election.cc:288] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a175ba33bac437bac96d63e54070297 (127.11.8.65:35315), ecd5818a9ded4cab8c333d87e5e52076 (127.11.8.66:35733)
I20260806 14:58:15.067085 13049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49cdaf4adbef4963abd02c08ee912b2e" candidate_uuid: "d514f0d51b9a445e873f6f37894d92b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a175ba33bac437bac96d63e54070297" is_pre_election: true
I20260806 14:58:15.067183 13049 raft_consensus.cc:2454] T 49cdaf4adbef4963abd02c08ee912b2e P 9a175ba33bac437bac96d63e54070297 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d514f0d51b9a445e873f6f37894d92b0 in term 0.
I20260806 14:58:15.067155 13244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49cdaf4adbef4963abd02c08ee912b2e" candidate_uuid: "d514f0d51b9a445e873f6f37894d92b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecd5818a9ded4cab8c333d87e5e52076" is_pre_election: true
I20260806 14:58:15.067286 13244 raft_consensus.cc:2454] T 49cdaf4adbef4963abd02c08ee912b2e P ecd5818a9ded4cab8c333d87e5e52076 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d514f0d51b9a445e873f6f37894d92b0 in term 0.
I20260806 14:58:15.067338 13339 leader_election.cc:302] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a175ba33bac437bac96d63e54070297, d514f0d51b9a445e873f6f37894d92b0; no voters:
I20260806 14:58:15.067453 13537 raft_consensus.cc:2781] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 14:58:15.067492 13537 raft_consensus.cc:482] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 14:58:15.067523 13537 raft_consensus.cc:3037] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:15.068372 13537 raft_consensus.cc:504] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } }
I20260806 14:58:15.068527 13537 leader_election.cc:288] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0 [CANDIDATE]: Term 1 election: Requested vote from peers 9a175ba33bac437bac96d63e54070297 (127.11.8.65:35315), ecd5818a9ded4cab8c333d87e5e52076 (127.11.8.66:35733)
I20260806 14:58:15.068671 13049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49cdaf4adbef4963abd02c08ee912b2e" candidate_uuid: "d514f0d51b9a445e873f6f37894d92b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a175ba33bac437bac96d63e54070297"
I20260806 14:58:15.068768 13049 raft_consensus.cc:3037] T 49cdaf4adbef4963abd02c08ee912b2e P 9a175ba33bac437bac96d63e54070297 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:15.068756 13244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49cdaf4adbef4963abd02c08ee912b2e" candidate_uuid: "d514f0d51b9a445e873f6f37894d92b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecd5818a9ded4cab8c333d87e5e52076"
I20260806 14:58:15.068848 13244 raft_consensus.cc:3037] T 49cdaf4adbef4963abd02c08ee912b2e P ecd5818a9ded4cab8c333d87e5e52076 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:15.069522 13049 raft_consensus.cc:2454] T 49cdaf4adbef4963abd02c08ee912b2e P 9a175ba33bac437bac96d63e54070297 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d514f0d51b9a445e873f6f37894d92b0 in term 1.
I20260806 14:58:15.069636 13244 raft_consensus.cc:2454] T 49cdaf4adbef4963abd02c08ee912b2e P ecd5818a9ded4cab8c333d87e5e52076 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d514f0d51b9a445e873f6f37894d92b0 in term 1.
I20260806 14:58:15.069746 13339 leader_election.cc:302] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a175ba33bac437bac96d63e54070297, d514f0d51b9a445e873f6f37894d92b0; no voters:
I20260806 14:58:15.069905 13537 raft_consensus.cc:2781] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:15.069972 13537 raft_consensus.cc:686] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0 [term 1 LEADER]: Becoming Leader. State: Replica: d514f0d51b9a445e873f6f37894d92b0, State: Running, Role: LEADER
I20260806 14:58:15.070032 13537 consensus_queue.cc:237] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } }
I20260806 14:58:15.070330 13577 tablet_replica.cc:439] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0: TxnStatusTablet state changed. Reason: New leader d514f0d51b9a445e873f6f37894d92b0. Latest consensus state: current_term: 1 leader_uuid: "d514f0d51b9a445e873f6f37894d92b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } } }
I20260806 14:58:15.070472 13577 tablet_replica.cc:442] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0: This TxnStatusTablet replica's current role is: LEADER
I20260806 14:58:15.070628 13598 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 14:58:15.070640 12846 catalog_manager.cc:5495] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d514f0d51b9a445e873f6f37894d92b0 (127.11.8.67). New cstate: current_term: 1 leader_uuid: "d514f0d51b9a445e873f6f37894d92b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } health_report { overall_health: HEALTHY } } }
I20260806 14:58:15.070779 13598 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 14:58:15.081261 13602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:15.078551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39808 (local address 127.11.8.67:36643)
0806 14:58:15.078721 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:15.078723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:15.078731 (+ 8us) server_negotiation.cc:407] Connection header received
0806 14:58:15.078765 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:15.078766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:15.078787 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:15.078856 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 14:58:15.078966 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:15.079495 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:15.080308 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:15.080622 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:15.080721 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:15.080917 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 14:58:15.081025 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 14:58:15.081100 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:15.081155 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 14:58:15.081178 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
I20260806 14:58:15.081295 13591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:15.078529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:36643 (local address 127.0.0.1:39808)
0806 14:58:15.078592 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:15.078604 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:15.078652 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:15.078862 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:15.078863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:15.078864 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 14:58:15.078914 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:15.078917 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:15.079596 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:15.079598 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:15.080219 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:15.080221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:15.080723 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:15.080724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:15.080787 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:15.080810 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 14:58:15.081098 (+ 288us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 14:58:15.081108 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 14:58:15.081154 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 14:58:15.081216 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260806 14:58:15.082535 13049 raft_consensus.cc:1261] T 49cdaf4adbef4963abd02c08ee912b2e P 9a175ba33bac437bac96d63e54070297 [term 1 FOLLOWER]: Refusing update from remote peer d514f0d51b9a445e873f6f37894d92b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:15.082660 13244 raft_consensus.cc:1261] T 49cdaf4adbef4963abd02c08ee912b2e P ecd5818a9ded4cab8c333d87e5e52076 [term 1 FOLLOWER]: Refusing update from remote peer d514f0d51b9a445e873f6f37894d92b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:15.082937 13537 consensus_queue.cc:1035] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:15.083250 13537 consensus_queue.cc:1035] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:15.083734 13541 tablet_replica.cc:439] T 49cdaf4adbef4963abd02c08ee912b2e P ecd5818a9ded4cab8c333d87e5e52076: TxnStatusTablet state changed. Reason: New leader d514f0d51b9a445e873f6f37894d92b0. Latest consensus state: current_term: 1 leader_uuid: "d514f0d51b9a445e873f6f37894d92b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } } }
I20260806 14:58:15.083755 13538 tablet_replica.cc:439] T 49cdaf4adbef4963abd02c08ee912b2e P 9a175ba33bac437bac96d63e54070297: TxnStatusTablet state changed. Reason: New leader d514f0d51b9a445e873f6f37894d92b0. Latest consensus state: current_term: 1 leader_uuid: "d514f0d51b9a445e873f6f37894d92b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } } }
I20260806 14:58:15.083827 13541 tablet_replica.cc:442] T 49cdaf4adbef4963abd02c08ee912b2e P ecd5818a9ded4cab8c333d87e5e52076: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 14:58:15.083841 13538 tablet_replica.cc:442] T 49cdaf4adbef4963abd02c08ee912b2e P 9a175ba33bac437bac96d63e54070297: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 14:58:15.084707 13577 tablet_replica.cc:439] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d514f0d51b9a445e873f6f37894d92b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } } }
I20260806 14:58:15.084782 13577 tablet_replica.cc:442] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0: This TxnStatusTablet replica's current role is: LEADER
I20260806 14:58:15.085139 13577 tablet_replica.cc:439] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d514f0d51b9a445e873f6f37894d92b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } } }
I20260806 14:58:15.085215 13577 tablet_replica.cc:442] T 49cdaf4adbef4963abd02c08ee912b2e P d514f0d51b9a445e873f6f37894d92b0: This TxnStatusTablet replica's current role is: LEADER
I20260806 14:58:15.085573 13604 mvcc.cc:205] Tried to move back new op lower bound from 7315571896656007168 to 7315571896607162368. Current Snapshot: MvccSnapshot[applied={T|T < 7315571896656007168}]
I20260806 14:58:15.085678 13541 tablet_replica.cc:439] T 49cdaf4adbef4963abd02c08ee912b2e P ecd5818a9ded4cab8c333d87e5e52076: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d514f0d51b9a445e873f6f37894d92b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } } }
I20260806 14:58:15.085690 13538 tablet_replica.cc:439] T 49cdaf4adbef4963abd02c08ee912b2e P 9a175ba33bac437bac96d63e54070297: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d514f0d51b9a445e873f6f37894d92b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 } } peers { permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 } } peers { permanent_uuid: "d514f0d51b9a445e873f6f37894d92b0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36643 } } }
I20260806 14:58:15.085747 13541 tablet_replica.cc:442] T 49cdaf4adbef4963abd02c08ee912b2e P ecd5818a9ded4cab8c333d87e5e52076: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 14:58:15.085768 13538 tablet_replica.cc:442] T 49cdaf4adbef4963abd02c08ee912b2e P 9a175ba33bac437bac96d63e54070297: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 14:58:15.090435 13602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:15.088076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39816 (local address 127.11.8.67:36643)
0806 14:58:15.088118 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:15.088120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:15.088208 (+ 88us) server_negotiation.cc:407] Connection header received
0806 14:58:15.088271 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:15.088272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:15.088282 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:15.088336 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 14:58:15.088476 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:15.088949 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:15.089704 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:15.089908 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:15.089985 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:15.090141 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 14:58:15.090174 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 14:58:15.090255 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:15.090316 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 14:58:15.090337 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 14:58:15.095505 13602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:15.092869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:52879 (local address 127.11.8.67:36643)
0806 14:58:15.092906 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:15.092907 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:15.093052 (+ 145us) server_negotiation.cc:407] Connection header received
0806 14:58:15.093107 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:15.093108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:15.093116 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:15.093162 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 14:58:15.093304 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:15.093718 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:15.094857 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:15.095053 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:15.095111 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:15.095285 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 14:58:15.095307 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 14:58:15.095342 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:15.095394 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 14:58:15.095409 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 14:58:15.095572 13616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:15.092814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:36643 (local address 127.11.8.67:52879)
0806 14:58:15.092976 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:15.092986 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:15.093052 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:15.093166 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:15.093167 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:15.093168 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 14:58:15.093240 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:15.093241 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:15.093793 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:15.093794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:15.094783 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:15.094786 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:15.095114 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:15.095115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:15.095192 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:15.095215 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 14:58:15.095344 (+ 129us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 14:58:15.095345 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 14:58:15.095393 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 14:58:15.095434 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
I20260806 14:58:15.097856 13048 raft_consensus.cc:1261] T 39a0ce0486754517b54af9f0f2614d0e P 9a175ba33bac437bac96d63e54070297 [term 1 FOLLOWER]: Refusing update from remote peer d514f0d51b9a445e873f6f37894d92b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:15.097856 13244 raft_consensus.cc:1261] T 39a0ce0486754517b54af9f0f2614d0e P ecd5818a9ded4cab8c333d87e5e52076 [term 1 FOLLOWER]: Refusing update from remote peer d514f0d51b9a445e873f6f37894d92b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:15.098047 13577 consensus_queue.cc:1035] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecd5818a9ded4cab8c333d87e5e52076" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:15.098131 13537 consensus_queue.cc:1035] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a175ba33bac437bac96d63e54070297" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 35315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:15.110951 12845 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:58:15.111047 12845 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:58:15.111702 12845 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 76641eb070df4b3882f0487226bc0962: Sending DeleteTablet for 3 replicas of tablet 39a0ce0486754517b54af9f0f2614d0e
I20260806 14:58:15.111961 13026 tablet_service.cc:1514] Processing DeleteTablet for tablet 39a0ce0486754517b54af9f0f2614d0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:15 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:32770
I20260806 14:58:15.112000 13219 tablet_service.cc:1514] Processing DeleteTablet for tablet 39a0ce0486754517b54af9f0f2614d0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:15 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:58288
I20260806 14:58:15.112200 13625 tablet_replica.cc:330] T 39a0ce0486754517b54af9f0f2614d0e P 9a175ba33bac437bac96d63e54070297: stopping tablet replica
I20260806 14:58:15.112206 13393 tablet_service.cc:1514] Processing DeleteTablet for tablet 39a0ce0486754517b54af9f0f2614d0e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:15 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:39796
I20260806 14:58:15.112311 13625 raft_consensus.cc:2229] T 39a0ce0486754517b54af9f0f2614d0e P 9a175ba33bac437bac96d63e54070297 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:58:15.112399 13629 tablet_replica.cc:330] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0: stopping tablet replica
I20260806 14:58:15.112417 13625 raft_consensus.cc:2258] T 39a0ce0486754517b54af9f0f2614d0e P 9a175ba33bac437bac96d63e54070297 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:15.112610 13629 raft_consensus.cc:2229] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0 [term 1 LEADER]: Raft consensus shutting down.
I20260806 14:58:15.112660 13625 ts_tablet_manager.cc:1905] T 39a0ce0486754517b54af9f0f2614d0e P 9a175ba33bac437bac96d63e54070297: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:15.112771 13629 raft_consensus.cc:2258] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:15.113049 13629 ts_tablet_manager.cc:1905] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:15.113128 13627 tablet_replica.cc:330] T 39a0ce0486754517b54af9f0f2614d0e P ecd5818a9ded4cab8c333d87e5e52076: stopping tablet replica
I20260806 14:58:15.113245 13627 raft_consensus.cc:2229] T 39a0ce0486754517b54af9f0f2614d0e P ecd5818a9ded4cab8c333d87e5e52076 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:58:15.113328 13627 pending_rounds.cc:62] T 39a0ce0486754517b54af9f0f2614d0e P ecd5818a9ded4cab8c333d87e5e52076: Trying to abort 2 pending ops.
I20260806 14:58:15.113389 13627 pending_rounds.cc:68] T 39a0ce0486754517b54af9f0f2614d0e P ecd5818a9ded4cab8c333d87e5e52076: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7315571896719212544 op_type: PARTICIPANT_OP participant_request { tablet_id: "39a0ce0486754517b54af9f0f2614d0e" op { type: BEGIN_TXN txn_id: 0 } }
I20260806 14:58:15.113576 13627 pending_rounds.cc:68] T 39a0ce0486754517b54af9f0f2614d0e P ecd5818a9ded4cab8c333d87e5e52076: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315571896159039488 op_type: NO_OP noop_request { }
I20260806 14:58:15.113703 13627 raft_consensus.cc:2866] T 39a0ce0486754517b54af9f0f2614d0e P ecd5818a9ded4cab8c333d87e5e52076 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260806 14:58:15.113796 13627 raft_consensus.cc:2258] T 39a0ce0486754517b54af9f0f2614d0e P ecd5818a9ded4cab8c333d87e5e52076 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:15.114710 13625 ts_tablet_manager.cc:1918] T 39a0ce0486754517b54af9f0f2614d0e P 9a175ba33bac437bac96d63e54070297: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 14:58:15.114745 13629 ts_tablet_manager.cc:1918] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
W20260806 14:58:15.114763 13622 consensus_queue.cc:1159] T 39a0ce0486754517b54af9f0f2614d0e P ecd5818a9ded4cab8c333d87e5e52076 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "ecd5818a9ded4cab8c333d87e5e52076" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
I20260806 14:58:15.114802 13629 log.cc:1196] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-2/wal/wals/39a0ce0486754517b54af9f0f2614d0e
I20260806 14:58:15.114815 13625 log.cc:1196] T 39a0ce0486754517b54af9f0f2614d0e P 9a175ba33bac437bac96d63e54070297: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-0/wal/wals/39a0ce0486754517b54af9f0f2614d0e
I20260806 14:58:15.115083 13629 ts_tablet_manager.cc:1939] T 39a0ce0486754517b54af9f0f2614d0e P d514f0d51b9a445e873f6f37894d92b0: Deleting consensus metadata
I20260806 14:58:15.115113 13625 ts_tablet_manager.cc:1939] T 39a0ce0486754517b54af9f0f2614d0e P 9a175ba33bac437bac96d63e54070297: Deleting consensus metadata
I20260806 14:58:15.115339 13627 ts_tablet_manager.cc:1905] T 39a0ce0486754517b54af9f0f2614d0e P ecd5818a9ded4cab8c333d87e5e52076: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:15.115480 12829 catalog_manager.cc:4841] TS 9a175ba33bac437bac96d63e54070297 (127.11.8.65:35315): tablet 39a0ce0486754517b54af9f0f2614d0e (table test-table [id=e6ad0e5b5286475b98bc8139aa9fe7d7]) successfully deleted
I20260806 14:58:15.115480 12831 catalog_manager.cc:4841] TS d514f0d51b9a445e873f6f37894d92b0 (127.11.8.67:36643): tablet 39a0ce0486754517b54af9f0f2614d0e (table test-table [id=e6ad0e5b5286475b98bc8139aa9fe7d7]) successfully deleted
I20260806 14:58:15.116593 13627 ts_tablet_manager.cc:1918] T 39a0ce0486754517b54af9f0f2614d0e P ecd5818a9ded4cab8c333d87e5e52076: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 14:58:15.116659 13627 log.cc:1196] T 39a0ce0486754517b54af9f0f2614d0e P ecd5818a9ded4cab8c333d87e5e52076: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786028287744255-11297-0/minicluster-data/ts-1/wal/wals/39a0ce0486754517b54af9f0f2614d0e
I20260806 14:58:15.116907 13627 ts_tablet_manager.cc:1939] T 39a0ce0486754517b54af9f0f2614d0e P ecd5818a9ded4cab8c333d87e5e52076: Deleting consensus metadata
I20260806 14:58:15.117226 12831 catalog_manager.cc:4841] TS ecd5818a9ded4cab8c333d87e5e52076 (127.11.8.66:35733): tablet 39a0ce0486754517b54af9f0f2614d0e (table test-table [id=e6ad0e5b5286475b98bc8139aa9fe7d7]) successfully deleted
Aug 06 14:58:15 dist-test-slave-68xk krb5kdc[12789](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028294, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.8.65@KRBTEST.COM
I20260806 14:58:15.123775 13524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:15.114081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32776 (local address 127.11.8.65:35315)
0806 14:58:15.114119 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:15.114121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:15.114367 (+ 246us) server_negotiation.cc:407] Connection header received
0806 14:58:15.114499 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:15.114500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:15.114511 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:15.114543 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:15.115478 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:15.116007 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:15.116878 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:15.117148 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:15.117230 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:15.120563 (+ 3333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:15.120572 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:15.120573 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:15.120580 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:15.122464 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:15.122843 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:15.122847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:15.122847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:15.122915 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:15.123108 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:15.123109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:15.123109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:15.123280 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:15.123419 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:15.123618 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 14:58:15.123696 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 14:58:15.124050 13105 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32776
I20260806 14:58:15.131047 13592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:15.125109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52406 (local address 127.11.8.126:44839)
0806 14:58:15.125153 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:15.125155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:15.125197 (+ 42us) server_negotiation.cc:407] Connection header received
0806 14:58:15.125258 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:15.125259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:15.125268 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:15.125308 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:15.125919 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:15.126391 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:15.127128 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:15.127343 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:15.127415 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:15.128570 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:15.128572 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:15.128572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:15.128579 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:15.129916 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:15.130264 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:15.130265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:15.130266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:15.130323 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:15.130532 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:15.130533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:15.130534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:15.130643 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:15.130760 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:15.130866 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 14:58:15.130948 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 14:58:15.131382 12845 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52406
I20260806 14:58:15.132009 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 12949
I20260806 14:58:15.146859 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 13125
I20260806 14:58:15.158342 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 13317
I20260806 14:58:15.166061 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 12809
2026-08-06T14:58:15Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2755 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 14:58:15 dist-test-slave-68xk krb5kdc[13639](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 14:58:15 dist-test-slave-68xk krb5kdc[13639](info): set up 2 sockets
Aug 06 14:58:15 dist-test-slave-68xk krb5kdc[13639](info): commencing operation
krb5kdc: starting...
W20260806 14:58:16.403545 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 14:58:16 dist-test-slave-68xk krb5kdc[13639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T14:58:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T14:58:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 14:58:16.529274 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:39323
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:36981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:39323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:16.612171 13661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:16.612417 13661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:16.612455 13661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:16.613595 13661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 14:58:16.613667 13661 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 14:58:16.613699 13661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:16.613729 13661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:16.613759 13661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 14:58:16.613790 13661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 14:58:16.615269 13661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:36981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:39323
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:39323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:16.615551 13661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:16.615794 13661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 14:58:16.620857 13661 server_base.cc:1034] running on GCE node
W20260806 14:58:16.620815 13669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:16.620851 13668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:16.620816 13672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:16.621292 13661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:16.621603 13661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:16.622740 13661 hybrid_clock.cc:648] HybridClock initialized: now 1786028296622729 us; error 31 us; skew 500 ppm
Aug 06 14:58:16 dist-test-slave-68xk krb5kdc[13639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028296, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:16.625305 13661 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 14:58:16.625876 13661 webserver.cc:460] Webserver started at http://127.11.8.126:46447/ using document root <none> and password file <none>
I20260806 14:58:16.626116 13661 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:16.626164 13661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:16.626305 13661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:16.627429 13661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "09dc9af5f18845a3aab211a08457af34"
format_stamp: "Formatted at 2026-08-06 14:58:16 on dist-test-slave-68xk"
server_key: "f2ec16a0b20449cf339e89a9ece34b8a"
server_key_iv: "bc16e7a0eb9043b9e8516f04252e82a5"
server_key_version: "encryptionkey@0"
I20260806 14:58:16.627815 13661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "09dc9af5f18845a3aab211a08457af34"
format_stamp: "Formatted at 2026-08-06 14:58:16 on dist-test-slave-68xk"
server_key: "f2ec16a0b20449cf339e89a9ece34b8a"
server_key_iv: "bc16e7a0eb9043b9e8516f04252e82a5"
server_key_version: "encryptionkey@0"
I20260806 14:58:16.629226 13661 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 14:58:16.630033 13684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:16.630244 13661 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:16.630352 13661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "09dc9af5f18845a3aab211a08457af34"
format_stamp: "Formatted at 2026-08-06 14:58:16 on dist-test-slave-68xk"
server_key: "f2ec16a0b20449cf339e89a9ece34b8a"
server_key_iv: "bc16e7a0eb9043b9e8516f04252e82a5"
server_key_version: "encryptionkey@0"
I20260806 14:58:16.630451 13661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:16.646567 13661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:16.648744 13661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:16.648940 13661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:16.654376 13661 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:39323
I20260806 14:58:16.654480 13775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:39323 every 8 connection(s)
I20260806 14:58:16.654878 13661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 14:58:16.655624 13776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:16.657976 13776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09dc9af5f18845a3aab211a08457af34: Bootstrap starting.
I20260806 14:58:16.658639 13776 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09dc9af5f18845a3aab211a08457af34: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:16.658951 13776 log.cc:824] T 00000000000000000000000000000000 P 09dc9af5f18845a3aab211a08457af34: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:16.659698 13776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09dc9af5f18845a3aab211a08457af34: No bootstrap required, opened a new log
I20260806 14:58:16.662015 13776 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09dc9af5f18845a3aab211a08457af34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09dc9af5f18845a3aab211a08457af34" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 39323 } }
I20260806 14:58:16.662164 13776 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09dc9af5f18845a3aab211a08457af34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:16.662189 13776 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09dc9af5f18845a3aab211a08457af34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09dc9af5f18845a3aab211a08457af34, State: Initialized, Role: FOLLOWER
I20260806 14:58:16.662366 13776 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09dc9af5f18845a3aab211a08457af34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09dc9af5f18845a3aab211a08457af34" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 39323 } }
I20260806 14:58:16.662495 13776 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09dc9af5f18845a3aab211a08457af34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 14:58:16.662557 13776 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09dc9af5f18845a3aab211a08457af34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 14:58:16.662602 13776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09dc9af5f18845a3aab211a08457af34 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:16.663177 13776 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09dc9af5f18845a3aab211a08457af34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09dc9af5f18845a3aab211a08457af34" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 39323 } }
I20260806 14:58:16.663280 13776 leader_election.cc:302] T 00000000000000000000000000000000 P 09dc9af5f18845a3aab211a08457af34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09dc9af5f18845a3aab211a08457af34; no voters:
I20260806 14:58:16.663468 13776 leader_election.cc:288] T 00000000000000000000000000000000 P 09dc9af5f18845a3aab211a08457af34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 14:58:16.663522 13779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09dc9af5f18845a3aab211a08457af34 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:16.663805 13779 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09dc9af5f18845a3aab211a08457af34 [term 1 LEADER]: Becoming Leader. State: Replica: 09dc9af5f18845a3aab211a08457af34, State: Running, Role: LEADER
I20260806 14:58:16.663785 13776 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09dc9af5f18845a3aab211a08457af34 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 14:58:16.664036 13779 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09dc9af5f18845a3aab211a08457af34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09dc9af5f18845a3aab211a08457af34" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 39323 } }
I20260806 14:58:16.664422 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 13661
I20260806 14:58:16.664520 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 14:58:16.664460 13782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09dc9af5f18845a3aab211a08457af34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09dc9af5f18845a3aab211a08457af34. Latest consensus state: current_term: 1 leader_uuid: "09dc9af5f18845a3aab211a08457af34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09dc9af5f18845a3aab211a08457af34" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 39323 } } }
I20260806 14:58:16.664471 13781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09dc9af5f18845a3aab211a08457af34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09dc9af5f18845a3aab211a08457af34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09dc9af5f18845a3aab211a08457af34" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 39323 } } }
I20260806 14:58:16.664561 13782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09dc9af5f18845a3aab211a08457af34 [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:16.664569 13781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09dc9af5f18845a3aab211a08457af34 [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:16.664733 11297 external_mini_cluster.cc:1428] Setting key d8c63c8a982e63e519b4a383c6c961a0
I20260806 14:58:16.664811 13790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 14:58:16.665410 13790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 14:58:16.667080 13790 catalog_manager.cc:1347] Generated new cluster ID: 3739875838234deabf52d37f45dd895d
I20260806 14:58:16.667146 13790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 14:58:16 dist-test-slave-68xk krb5kdc[13639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:16.674186 13790 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 14:58:16.675045 13790 catalog_manager.cc:1504] Loading token signing keys...
I20260806 14:58:16.678095 13803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:16.665423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58314 (local address 127.11.8.126:39323)
0806 14:58:16.665594 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:16.665605 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:16.666073 (+ 468us) server_negotiation.cc:407] Connection header received
0806 14:58:16.666248 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:16.666256 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:16.666319 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:16.666476 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:16.667366 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:16.668729 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:16.669829 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:16.670482 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:16.670583 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:16.673862 (+ 3279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:16.673867 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:16.673867 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:16.673886 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:16.676128 (+ 2242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:16.676565 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:16.676568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:16.676569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:16.676693 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:16.677051 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:16.677051 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:16.677052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:16.677344 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:16.677489 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:16.677679 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 14:58:16.677838 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":58,"threads_started":1}
I20260806 14:58:16.679823 13790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09dc9af5f18845a3aab211a08457af34: Generated new TSK 0
I20260806 14:58:16.680012 13790 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
I20260806 14:58:16.713166 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:0
--local_ip_for_outbound_sockets=127.11.8.65
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:39323
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:36981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:16.797163 13812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:16.797369 13812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:16.797394 13812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:16.798621 13812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:16.798666 13812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:16.798686 13812 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 14:58:16.798698 13812 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 14:58:16.798738 13812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 14:58:16.800196 13812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:36981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.8.126:39323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:16.800441 13812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:16.800685 13812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:16.805845 13826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:16.805980 13812 server_base.cc:1034] running on GCE node
W20260806 14:58:16.805858 13824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:16.805853 13823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:16.806325 13812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:16.806639 13812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:16.807809 13812 hybrid_clock.cc:648] HybridClock initialized: now 1786028296807802 us; error 35 us; skew 500 ppm
Aug 06 14:58:16 dist-test-slave-68xk krb5kdc[13639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028296, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:16.810513 13812 init.cc:377] Logged in from keytab as kudu/127.11.8.65@KRBTEST.COM (short username kudu)
I20260806 14:58:16.810935 13812 webserver.cc:460] Webserver started at http://127.11.8.65:35591/ using document root <none> and password file <none>
I20260806 14:58:16.811218 13812 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:16.811424 13812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:16.811589 13812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:16.812646 13812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-0/data/instance:
uuid: "2b9be7562599461aac701ceb1544a926"
format_stamp: "Formatted at 2026-08-06 14:58:16 on dist-test-slave-68xk"
server_key: "aeec7d552849a4fd9de94f087fd2eb3a"
server_key_iv: "bb0226b8628934864f4d0e2f49dd3caa"
server_key_version: "encryptionkey@0"
I20260806 14:58:16.813016 13812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance:
uuid: "2b9be7562599461aac701ceb1544a926"
format_stamp: "Formatted at 2026-08-06 14:58:16 on dist-test-slave-68xk"
server_key: "aeec7d552849a4fd9de94f087fd2eb3a"
server_key_iv: "bb0226b8628934864f4d0e2f49dd3caa"
server_key_version: "encryptionkey@0"
I20260806 14:58:16.814392 13812 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 14:58:16.815263 13833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:16.815452 13812 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:16.815555 13812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "2b9be7562599461aac701ceb1544a926"
format_stamp: "Formatted at 2026-08-06 14:58:16 on dist-test-slave-68xk"
server_key: "aeec7d552849a4fd9de94f087fd2eb3a"
server_key_iv: "bb0226b8628934864f4d0e2f49dd3caa"
server_key_version: "encryptionkey@0"
I20260806 14:58:16.815639 13812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:16.833865 13812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:16.836774 13812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:16.836946 13812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:16.837482 13812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:16.837558 13812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:16.837601 13812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:16.837635 13812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:16.844611 13812 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:36799
I20260806 14:58:16.844683 13989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:36799 every 8 connection(s)
Aug 06 14:58:16 dist-test-slave-68xk krb5kdc[13639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028296, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:16.845047 13812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
I20260806 14:58:16.848237 13803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:16.838008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:36161 (local address 127.11.8.126:39323)
0806 14:58:16.838052 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:16.838054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:16.838463 (+ 409us) server_negotiation.cc:407] Connection header received
0806 14:58:16.838971 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:16.838973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:16.838986 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:16.839069 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:16.839920 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:16.840803 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:16.842386 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:16.842754 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:16.842850 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:16.845669 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:16.845670 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:16.845671 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:16.845677 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:16.846927 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:16.847359 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:16.847360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:16.847360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:16.847415 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:16.847643 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:16.847643 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:16.847644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:16.847741 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:16.847798 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:16.848100 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 14:58:16.848174 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 14:58:16.848549 13845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:16.838114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:39323 (local address 127.11.8.65:36161)
0806 14:58:16.838312 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:16.838347 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:16.838864 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:16.839138 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:16.839145 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:16.839492 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:16.839781 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:16.839787 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:16.840914 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:16.840916 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:16.842281 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:16.842284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:16.842848 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:16.842849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:16.842949 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:16.843513 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:16.843523 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:16.845568 (+ 2045us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:16.847008 (+ 1440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:16.847015 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:16.847017 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:16.847288 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:16.847476 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:16.847476 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:16.847477 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:16.847586 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:16.847830 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:16.847837 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:16.847972 (+ 135us) client_negotiation.cc:769] Sending connection context
0806 14:58:16.848077 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 14:58:16.848249 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":114,"threads_started":1}
I20260806 14:58:16.849114 13991 heartbeater.cc:347] Connected to a master server at 127.11.8.126:39323
I20260806 14:58:16.849238 13991 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:16.849428 13991 heartbeater.cc:511] Master 127.11.8.126:39323 requested a full tablet report, sending...
I20260806 14:58:16.849424 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 13812
I20260806 14:58:16.849534 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance
I20260806 14:58:16.849767 11297 external_mini_cluster.cc:1428] Setting key 84c6577f02638ed7b7c3652255f8c110
I20260806 14:58:16.850086 13703 ts_manager.cc:194] Registered new tserver with Master: 2b9be7562599461aac701ceb1544a926 (127.11.8.65:36799)
I20260806 14:58:16.851143 13703 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.65@KRBTEST.COM'} at 127.11.8.65:36161
I20260806 14:58:16.856797 13803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:16.850199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:60847 (local address 127.11.8.126:39323)
0806 14:58:16.850240 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:16.850242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:16.850512 (+ 270us) server_negotiation.cc:407] Connection header received
0806 14:58:16.850587 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:16.850588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:16.850599 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:16.850649 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:16.851026 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:16.851723 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:16.852511 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:16.852797 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:16.852896 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:16.853942 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:16.853943 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:16.853944 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:16.853952 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:16.855511 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:16.855932 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:16.855933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:16.855934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:16.855995 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:16.856238 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:16.856239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:16.856240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:16.856360 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:16.856445 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:16.856626 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 14:58:16.856702 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 14:58:16.856809 13998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:16.850152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:39323 (local address 127.11.8.65:60847)
0806 14:58:16.850448 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:16.850460 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:16.850526 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:16.850653 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:16.850653 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:16.850880 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:16.850943 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:16.850945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:16.851826 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:16.851827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:16.852392 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:16.852393 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:16.852953 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:16.852954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:16.853032 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:16.853521 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:16.853525 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:16.853831 (+ 306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:16.855641 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:16.855643 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:16.855644 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:16.855847 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:16.856085 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:16.856086 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:16.856087 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:16.856162 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:16.856437 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:16.856438 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:16.856486 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 14:58:16.856583 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 14:58:16.856702 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":177,"threads_started":1}
WARNING: no policy specified for kudu/127.11.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
I20260806 14:58:16.877892 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:0
--local_ip_for_outbound_sockets=127.11.8.66
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:39323
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:36981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:16.961035 14004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:16.961246 14004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:16.961272 14004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:16.962436 14004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:16.962474 14004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:16.962495 14004 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 14:58:16.962507 14004 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 14:58:16.962524 14004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 14:58:16.964015 14004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:36981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.8.126:39323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:16.964231 14004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:16.964514 14004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:16.970012 14012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:16.970130 14004 server_base.cc:1034] running on GCE node
W20260806 14:58:16.970000 14011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:16.970252 14014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:16.970543 14004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:16.970844 14004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:16.972007 14004 hybrid_clock.cc:648] HybridClock initialized: now 1786028296971988 us; error 31 us; skew 500 ppm
Aug 06 14:58:16 dist-test-slave-68xk krb5kdc[13639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028296, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:16.974473 14004 init.cc:377] Logged in from keytab as kudu/127.11.8.66@KRBTEST.COM (short username kudu)
I20260806 14:58:16.974924 14004 webserver.cc:460] Webserver started at http://127.11.8.66:39849/ using document root <none> and password file <none>
I20260806 14:58:16.975157 14004 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:16.975227 14004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:16.975364 14004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:16.976383 14004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-1/data/instance:
uuid: "c3bf26a2ee61406f9eca9b224c603adc"
format_stamp: "Formatted at 2026-08-06 14:58:16 on dist-test-slave-68xk"
server_key: "b6e5a047b96af59fea546abc10cb4b94"
server_key_iv: "9a39055ac4eed13cda724affecff17a8"
server_key_version: "encryptionkey@0"
I20260806 14:58:16.976835 14004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance:
uuid: "c3bf26a2ee61406f9eca9b224c603adc"
format_stamp: "Formatted at 2026-08-06 14:58:16 on dist-test-slave-68xk"
server_key: "b6e5a047b96af59fea546abc10cb4b94"
server_key_iv: "9a39055ac4eed13cda724affecff17a8"
server_key_version: "encryptionkey@0"
I20260806 14:58:16.978210 14004 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 14:58:16.979108 14022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:16.979288 14004 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 14:58:16.979398 14004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "c3bf26a2ee61406f9eca9b224c603adc"
format_stamp: "Formatted at 2026-08-06 14:58:16 on dist-test-slave-68xk"
server_key: "b6e5a047b96af59fea546abc10cb4b94"
server_key_iv: "9a39055ac4eed13cda724affecff17a8"
server_key_version: "encryptionkey@0"
I20260806 14:58:16.979488 14004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:16.995554 14004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:16.997701 14004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:16.997879 14004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:16.998538 14004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:16.998603 14004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:16.998636 14004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:16.998682 14004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 14:58:16 dist-test-slave-68xk krb5kdc[13639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028296, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:17.006789 14004 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:41741
I20260806 14:58:17.007113 14178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:41741 every 8 connection(s)
I20260806 14:58:17.007691 14004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
I20260806 14:58:17.010975 13803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:16.999232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:48059 (local address 127.11.8.126:39323)
0806 14:58:16.999293 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:16.999296 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:16.999568 (+ 272us) server_negotiation.cc:407] Connection header received
0806 14:58:17.000069 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:17.000071 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:17.000087 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:17.000174 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:17.001131 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.002076 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.003777 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.004166 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.004265 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.006997 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:17.006999 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:17.007000 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:17.007009 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:17.008938 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:17.009567 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:17.009568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:17.009568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:17.009629 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:17.009985 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:17.009986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:17.009986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:17.010231 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:17.010308 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:17.010743 (+ 435us) server_negotiation.cc:299] Negotiation successful
0806 14:58:17.010851 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 14:58:17.011175 14032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:16.999200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:39323 (local address 127.11.8.66:48059)
0806 14:58:16.999442 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:16.999480 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:16.999959 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:17.000321 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:17.000325 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:17.000720 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:17.001008 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.001015 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.002208 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.002209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.003664 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.003667 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.004281 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.004283 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.004399 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.005011 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:17.005035 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:17.006884 (+ 1849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:17.009069 (+ 2185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:17.009076 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:17.009079 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:17.009460 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:17.009710 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:17.009711 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:17.009712 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:17.009877 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:17.010341 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:17.010348 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:17.010551 (+ 203us) client_negotiation.cc:769] Sending connection context
0806 14:58:17.010682 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 14:58:17.010872 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":131,"threads_started":1}
I20260806 14:58:17.011991 14179 heartbeater.cc:347] Connected to a master server at 127.11.8.126:39323
I20260806 14:58:17.012140 14179 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:17.012404 14179 heartbeater.cc:511] Master 127.11.8.126:39323 requested a full tablet report, sending...
I20260806 14:58:17.012888 13702 ts_manager.cc:194] Registered new tserver with Master: c3bf26a2ee61406f9eca9b224c603adc (127.11.8.66:41741)
I20260806 14:58:17.013432 13702 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.66@KRBTEST.COM'} at 127.11.8.66:48059
I20260806 14:58:17.013473 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 14004
I20260806 14:58:17.013587 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance
I20260806 14:58:17.013808 11297 external_mini_cluster.cc:1428] Setting key 9ccf8a6d9340dfb5c07e40963ae161be
I20260806 14:58:17.020846 13803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.013624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:52393 (local address 127.11.8.126:39323)
0806 14:58:17.013811 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:17.013813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:17.013821 (+ 8us) server_negotiation.cc:407] Connection header received
0806 14:58:17.013851 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:17.013852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:17.013859 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:17.013933 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:17.014396 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.015117 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.016319 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.016514 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.016572 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.017644 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:17.017646 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:17.017646 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:17.017654 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:17.019467 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:17.019932 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:17.019934 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:17.019934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:17.019994 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:17.020237 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:17.020238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:17.020238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:17.020396 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:17.020483 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:17.020654 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 14:58:17.020740 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168}
I20260806 14:58:17.020869 14188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.013247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:39323 (local address 127.11.8.66:52393)
0806 14:58:17.013637 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:17.013650 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:17.013715 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:17.013953 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:17.013954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:17.014210 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:17.014288 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.014291 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.015281 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.015282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.016233 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.016237 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.016600 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.016601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.016692 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.017223 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:17.017229 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:17.017559 (+ 330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:17.019598 (+ 2039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:17.019599 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:17.019599 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:17.019849 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:17.020081 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:17.020082 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:17.020083 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:17.020150 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:17.020473 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:17.020474 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:17.020526 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 14:58:17.020625 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 14:58:17.020742 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":111,"threads_started":1}
WARNING: no policy specified for kudu/127.11.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
I20260806 14:58:17.041563 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:0
--local_ip_for_outbound_sockets=127.11.8.67
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:39323
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:36981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:17.125327 14191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:17.125533 14191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:17.125558 14191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:17.126657 14191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:17.126693 14191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:17.126714 14191 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 14:58:17.126726 14191 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 14:58:17.126742 14191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 14:58:17.128114 14191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:36981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.8.126:39323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:17.128300 14191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:17.128479 14191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:17.133956 14198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:17.133957 14203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:17.133971 14200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:17.134603 14191 server_base.cc:1034] running on GCE node
I20260806 14:58:17.134817 14191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:17.135038 14191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:17.136210 14191 hybrid_clock.cc:648] HybridClock initialized: now 1786028297136188 us; error 49 us; skew 500 ppm
Aug 06 14:58:17 dist-test-slave-68xk krb5kdc[13639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028297, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:17.138957 14191 init.cc:377] Logged in from keytab as kudu/127.11.8.67@KRBTEST.COM (short username kudu)
I20260806 14:58:17.139415 14191 webserver.cc:460] Webserver started at http://127.11.8.67:34709/ using document root <none> and password file <none>
I20260806 14:58:17.139678 14191 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:17.139724 14191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:17.139876 14191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:17.141067 14191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-2/data/instance:
uuid: "66922775c78f4c65926895f9744e03a4"
format_stamp: "Formatted at 2026-08-06 14:58:17 on dist-test-slave-68xk"
server_key: "03cd92cb5700f9e9e66388afce8ffb23"
server_key_iv: "88a71f725f600c74fc7dc62304e7888d"
server_key_version: "encryptionkey@0"
I20260806 14:58:17.141441 14191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance:
uuid: "66922775c78f4c65926895f9744e03a4"
format_stamp: "Formatted at 2026-08-06 14:58:17 on dist-test-slave-68xk"
server_key: "03cd92cb5700f9e9e66388afce8ffb23"
server_key_iv: "88a71f725f600c74fc7dc62304e7888d"
server_key_version: "encryptionkey@0"
I20260806 14:58:17.142859 14191 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 14:58:17.143644 14211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:17.143882 14191 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:17.143958 14191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "66922775c78f4c65926895f9744e03a4"
format_stamp: "Formatted at 2026-08-06 14:58:17 on dist-test-slave-68xk"
server_key: "03cd92cb5700f9e9e66388afce8ffb23"
server_key_iv: "88a71f725f600c74fc7dc62304e7888d"
server_key_version: "encryptionkey@0"
I20260806 14:58:17.144048 14191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:17.168066 14191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:17.170125 14191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:17.170288 14191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:17.170850 14191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:17.170924 14191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:17.170974 14191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:17.171002 14191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:17.178390 14191 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:40001
I20260806 14:58:17.178563 14374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:40001 every 8 connection(s)
I20260806 14:58:17.178835 14191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
Aug 06 14:58:17 dist-test-slave-68xk krb5kdc[13639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028297, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:17.182468 13803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.171445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:37029 (local address 127.11.8.126:39323)
0806 14:58:17.171518 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:17.171536 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:17.171720 (+ 184us) server_negotiation.cc:407] Connection header received
0806 14:58:17.172291 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:17.172293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:17.172308 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:17.172368 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:17.173418 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.174291 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.176043 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.176267 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.176328 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.179534 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:17.179536 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:17.179536 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:17.179543 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:17.181025 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:17.181428 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:17.181428 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:17.181429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:17.181481 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:17.181733 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:17.181733 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:17.181733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:17.181840 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:17.181899 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:17.182210 (+ 311us) server_negotiation.cc:299] Negotiation successful
0806 14:58:17.182336 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 14:58:17.182561 14226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.171401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:39323 (local address 127.11.8.67:37029)
0806 14:58:17.171585 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:17.171633 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:17.172167 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:17.172492 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:17.172498 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:17.172914 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:17.173258 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.173275 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.174438 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.174439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.175943 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.175946 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.176335 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.176337 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.176496 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.177286 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:17.177297 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:17.179421 (+ 2124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:17.181112 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:17.181118 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:17.181120 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:17.181354 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:17.181560 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:17.181561 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:17.181561 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:17.181666 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:17.181928 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:17.181934 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:17.182074 (+ 140us) client_negotiation.cc:769] Sending connection context
0806 14:58:17.182169 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 14:58:17.182298 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":100,"threads_started":1}
I20260806 14:58:17.183270 14375 heartbeater.cc:347] Connected to a master server at 127.11.8.126:39323
I20260806 14:58:17.183357 14375 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:17.183548 14375 heartbeater.cc:511] Master 127.11.8.126:39323 requested a full tablet report, sending...
I20260806 14:58:17.183923 13702 ts_manager.cc:194] Registered new tserver with Master: 66922775c78f4c65926895f9744e03a4 (127.11.8.67:40001)
I20260806 14:58:17.184314 13702 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.67@KRBTEST.COM'} at 127.11.8.67:37029
I20260806 14:58:17.187152 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 14191
I20260806 14:58:17.187249 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance
I20260806 14:58:17.187440 11297 external_mini_cluster.cc:1428] Setting key 29e7b8e17d2ad3c3cc49a285e4a5d109
I20260806 14:58:17.188452 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 14:58:17.190507 13803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.184488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:39963 (local address 127.11.8.126:39323)
0806 14:58:17.184528 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:17.184531 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:17.184639 (+ 108us) server_negotiation.cc:407] Connection header received
0806 14:58:17.184709 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:17.184710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:17.184720 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:17.184787 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:17.185094 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.185569 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.186344 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.186594 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.186651 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.187849 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:17.187850 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:17.187850 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:17.187857 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:17.189318 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:17.189682 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:17.189684 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:17.189684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:17.189738 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:17.189952 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:17.189953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:17.189954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:17.190072 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:17.190148 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:17.190319 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 14:58:17.190394 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 14:58:17.190516 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.184447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:39323 (local address 127.11.8.67:39963)
0806 14:58:17.184582 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:17.184594 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:17.184658 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:17.184788 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:17.184789 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:17.184943 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:17.185000 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.185001 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.185680 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.185682 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.186253 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.186255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.186695 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.186696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.186759 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.187289 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:17.187295 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:17.187749 (+ 454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:17.189410 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:17.189411 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:17.189412 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:17.189603 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:17.189822 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:17.189823 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:17.189823 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:17.189887 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:17.190155 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:17.190156 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:17.190209 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 14:58:17.190293 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 14:58:17.190397 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
Aug 06 14:58:17 dist-test-slave-68xk krb5kdc[13639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 14:58:17 dist-test-slave-68xk krb5kdc[13639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:17.212925 13803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.205808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58330 (local address 127.11.8.126:39323)
0806 14:58:17.205884 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:17.205886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:17.205895 (+ 9us) server_negotiation.cc:407] Connection header received
0806 14:58:17.205950 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:17.205951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:17.205960 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:17.206013 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:17.206625 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.207103 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.207815 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.207989 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.208040 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.210273 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:17.210275 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:17.210275 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:17.210283 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:17.211705 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:17.212041 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:17.212042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:17.212042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:17.212154 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:17.212357 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:17.212359 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:17.212359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:17.212482 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:17.212587 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:17.212747 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 14:58:17.212823 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 14:58:17.214722 13702 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 14:58:17.216240 13702 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 14:58:17.222764 14398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.218585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:40001 (local address 127.0.0.1:44214)
0806 14:58:17.218769 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:17.218812 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:17.218893 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:17.219463 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:17.219464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:17.219465 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:17.219495 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.219497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.220728 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.220729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.221744 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.221746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.222439 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.222440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.222538 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.222555 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 14:58:17.222626 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 14:58:17.222684 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":102,"threads_started":1}
I20260806 14:58:17.222764 14396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.218585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:36799 (local address 127.0.0.1:40680)
0806 14:58:17.218839 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:17.218848 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:17.218893 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:17.219419 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:17.219420 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:17.219422 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:17.219479 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.219482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.220363 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.220364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.221552 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.221555 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.222547 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.222547 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.222614 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.222617 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 14:58:17.222669 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 14:58:17.222696 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":2,"thread_start_us":63,"threads_started":1}
I20260806 14:58:17.222921 14397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.218589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:41741 (local address 127.0.0.1:60438)
0806 14:58:17.219040 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:17.219049 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:17.219086 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:17.219339 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:17.219342 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:17.219344 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:17.219408 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.219410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.220771 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.220772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.221670 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.221679 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.222687 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.222689 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.222744 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.222748 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 14:58:17.222801 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 14:58:17.222842 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":113,"threads_started":1}
I20260806 14:58:17.222945 14401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.219016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40680 (local address 127.11.8.65:36799)
0806 14:58:17.219223 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:17.219228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:17.219240 (+ 12us) server_negotiation.cc:407] Connection header received
0806 14:58:17.219284 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:17.219286 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:17.219301 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:17.219560 (+ 259us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:17.219625 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.220273 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.221657 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.222401 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.222545 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.222612 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:17.222683 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 14:58:17.222729 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":79,"threads_started":1}
I20260806 14:58:17.223081 14399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.218770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44214 (local address 127.11.8.67:40001)
0806 14:58:17.219247 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:17.219252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:17.219261 (+ 9us) server_negotiation.cc:407] Connection header received
0806 14:58:17.219300 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:17.219303 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:17.219317 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:17.219397 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:17.219596 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.220637 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.221834 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.222329 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.222418 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.222503 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:17.222895 (+ 392us) server_negotiation.cc:299] Negotiation successful
0806 14:58:17.222950 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":113,"threads_started":1}
I20260806 14:58:17.223264 14400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.218871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60438 (local address 127.11.8.66:41741)
0806 14:58:17.219079 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:17.219085 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:17.219096 (+ 11us) server_negotiation.cc:407] Connection header received
0806 14:58:17.219160 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:17.219162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:17.219177 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:17.219250 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:17.219490 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.220686 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.221780 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.222537 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.222594 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.222641 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:17.223081 (+ 440us) server_negotiation.cc:299] Negotiation successful
0806 14:58:17.223125 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":79,"threads_started":1}
I20260806 14:58:17.223945 14270 tablet_service.cc:1467] Processing CreateTablet for tablet fdb2e9ccdc7e4f74ad1e26666c574fe3 (DEFAULT_TABLE table=test-table [id=529e8c0061f74a76bcc2e54d6360bf4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:17.223995 14085 tablet_service.cc:1467] Processing CreateTablet for tablet fdb2e9ccdc7e4f74ad1e26666c574fe3 (DEFAULT_TABLE table=test-table [id=529e8c0061f74a76bcc2e54d6360bf4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:17.224067 13901 tablet_service.cc:1467] Processing CreateTablet for tablet fdb2e9ccdc7e4f74ad1e26666c574fe3 (DEFAULT_TABLE table=test-table [id=529e8c0061f74a76bcc2e54d6360bf4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:17.224321 14270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdb2e9ccdc7e4f74ad1e26666c574fe3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:17.224363 14085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdb2e9ccdc7e4f74ad1e26666c574fe3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:17.224375 13901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdb2e9ccdc7e4f74ad1e26666c574fe3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:17.227052 14402 tablet_bootstrap.cc:492] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4: Bootstrap starting.
I20260806 14:58:17.227881 14402 tablet_bootstrap.cc:654] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:17.228266 14402 log.cc:824] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:17.228269 14403 tablet_bootstrap.cc:492] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 2b9be7562599461aac701ceb1544a926: Bootstrap starting.
I20260806 14:58:17.228865 14403 tablet_bootstrap.cc:654] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 2b9be7562599461aac701ceb1544a926: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:17.229117 14403 log.cc:824] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 2b9be7562599461aac701ceb1544a926: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:17.229180 14402 tablet_bootstrap.cc:492] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4: No bootstrap required, opened a new log
I20260806 14:58:17.229259 14402 ts_tablet_manager.cc:1397] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 14:58:17.229449 14404 tablet_bootstrap.cc:492] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P c3bf26a2ee61406f9eca9b224c603adc: Bootstrap starting.
I20260806 14:58:17.229995 14404 tablet_bootstrap.cc:654] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P c3bf26a2ee61406f9eca9b224c603adc: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:17.230294 14404 log.cc:824] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P c3bf26a2ee61406f9eca9b224c603adc: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:17.230832 14403 tablet_bootstrap.cc:492] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 2b9be7562599461aac701ceb1544a926: No bootstrap required, opened a new log
I20260806 14:58:17.230952 14403 ts_tablet_manager.cc:1397] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 2b9be7562599461aac701ceb1544a926: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 14:58:17.231107 14404 tablet_bootstrap.cc:492] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P c3bf26a2ee61406f9eca9b224c603adc: No bootstrap required, opened a new log
I20260806 14:58:17.231184 14404 ts_tablet_manager.cc:1397] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P c3bf26a2ee61406f9eca9b224c603adc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 14:58:17.231448 14402 raft_consensus.cc:348] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } }
I20260806 14:58:17.231616 14402 raft_consensus.cc:374] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:17.231662 14402 raft_consensus.cc:729] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66922775c78f4c65926895f9744e03a4, State: Initialized, Role: FOLLOWER
I20260806 14:58:17.231774 14402 consensus_queue.cc:260] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } }
I20260806 14:58:17.232095 14375 heartbeater.cc:503] Master 127.11.8.126:39323 was elected leader, sending a full tablet report...
I20260806 14:58:17.232082 14402 ts_tablet_manager.cc:1428] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 14:58:17.232997 14403 raft_consensus.cc:348] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 2b9be7562599461aac701ceb1544a926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } }
I20260806 14:58:17.233093 14404 raft_consensus.cc:348] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } }
I20260806 14:58:17.233156 14403 raft_consensus.cc:374] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 2b9be7562599461aac701ceb1544a926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:17.233243 14404 raft_consensus.cc:374] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:17.233242 14403 raft_consensus.cc:729] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 2b9be7562599461aac701ceb1544a926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b9be7562599461aac701ceb1544a926, State: Initialized, Role: FOLLOWER
I20260806 14:58:17.233284 14404 raft_consensus.cc:729] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3bf26a2ee61406f9eca9b224c603adc, State: Initialized, Role: FOLLOWER
I20260806 14:58:17.233359 14403 consensus_queue.cc:260] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 2b9be7562599461aac701ceb1544a926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } }
I20260806 14:58:17.233408 14404 consensus_queue.cc:260] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P c3bf26a2ee61406f9eca9b224c603adc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } }
I20260806 14:58:17.233641 14404 ts_tablet_manager.cc:1428] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P c3bf26a2ee61406f9eca9b224c603adc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 14:58:17.233641 14403 ts_tablet_manager.cc:1428] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 2b9be7562599461aac701ceb1544a926: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 14:58:17.233683 14179 heartbeater.cc:503] Master 127.11.8.126:39323 was elected leader, sending a full tablet report...
I20260806 14:58:17.233677 13991 heartbeater.cc:503] Master 127.11.8.126:39323 was elected leader, sending a full tablet report...
W20260806 14:58:17.258675 14180 tablet.cc:2369] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P c3bf26a2ee61406f9eca9b224c603adc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 14:58:17.345827 13992 tablet.cc:2369] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 2b9be7562599461aac701ceb1544a926: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 14:58:17.429603 14376 tablet.cc:2369] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 14:58:17.448729 14412 raft_consensus.cc:482] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 14:58:17.448904 14412 raft_consensus.cc:504] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } }
I20260806 14:58:17.449556 14412 leader_election.cc:288] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3bf26a2ee61406f9eca9b224c603adc (127.11.8.66:41741), 2b9be7562599461aac701ceb1544a926 (127.11.8.65:36799)
I20260806 14:58:17.455264 14226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.449881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:36799 (local address 127.11.8.67:42111)
0806 14:58:17.449983 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:17.450001 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:17.450073 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:17.450335 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:17.450344 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:17.450346 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:17.450470 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.450472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.451817 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.451818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.453759 (+ 1941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.453763 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.454822 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.454824 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.454973 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.454985 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 14:58:17.455089 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 14:58:17.455148 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":115}
I20260806 14:58:17.455286 14415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.449910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:41741 (local address 127.11.8.67:50387)
0806 14:58:17.450198 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:17.450212 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:17.450293 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:17.450546 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:17.450547 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:17.450549 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:17.450658 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.450660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.451817 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.451818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.453759 (+ 1941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.453763 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.454822 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.454824 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.454973 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.454986 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 14:58:17.455089 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 14:58:17.455172 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":169,"thread_start_us":184,"threads_started":1}
I20260806 14:58:17.455322 14400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.450001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:50387 (local address 127.11.8.66:41741)
0806 14:58:17.450076 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:17.450078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:17.450282 (+ 204us) server_negotiation.cc:407] Connection header received
0806 14:58:17.450439 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:17.450441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:17.450453 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:17.450566 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:17.450749 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.451696 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.453895 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.454673 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.454809 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.454822 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:17.455149 (+ 327us) server_negotiation.cc:299] Negotiation successful
0806 14:58:17.455222 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 14:58:17.455322 14401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.450030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:42111 (local address 127.11.8.65:36799)
0806 14:58:17.450112 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:17.450123 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:17.450133 (+ 10us) server_negotiation.cc:407] Connection header received
0806 14:58:17.450185 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:17.450186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:17.450200 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:17.450305 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:17.450584 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.451667 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.453895 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.454723 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.454817 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.454826 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:17.455149 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 14:58:17.455232 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 14:58:17.455920 14112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdb2e9ccdc7e4f74ad1e26666c574fe3" candidate_uuid: "66922775c78f4c65926895f9744e03a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3bf26a2ee61406f9eca9b224c603adc" is_pre_election: true
I20260806 14:58:17.455894 13926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdb2e9ccdc7e4f74ad1e26666c574fe3" candidate_uuid: "66922775c78f4c65926895f9744e03a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b9be7562599461aac701ceb1544a926" is_pre_election: true
I20260806 14:58:17.456144 14112 raft_consensus.cc:2454] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66922775c78f4c65926895f9744e03a4 in term 0.
I20260806 14:58:17.456144 13926 raft_consensus.cc:2454] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 2b9be7562599461aac701ceb1544a926 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66922775c78f4c65926895f9744e03a4 in term 0.
I20260806 14:58:17.456488 14212 leader_election.cc:302] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66922775c78f4c65926895f9744e03a4, c3bf26a2ee61406f9eca9b224c603adc; no voters:
I20260806 14:58:17.456682 14412 raft_consensus.cc:2781] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 14:58:17.456772 14412 raft_consensus.cc:482] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 14:58:17.456831 14412 raft_consensus.cc:3037] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:17.458223 14412 raft_consensus.cc:504] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } }
I20260806 14:58:17.458499 14412 leader_election.cc:288] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4 [CANDIDATE]: Term 1 election: Requested vote from peers c3bf26a2ee61406f9eca9b224c603adc (127.11.8.66:41741), 2b9be7562599461aac701ceb1544a926 (127.11.8.65:36799)
I20260806 14:58:17.458694 14112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdb2e9ccdc7e4f74ad1e26666c574fe3" candidate_uuid: "66922775c78f4c65926895f9744e03a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3bf26a2ee61406f9eca9b224c603adc"
I20260806 14:58:17.458853 14112 raft_consensus.cc:3037] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:17.458914 13926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdb2e9ccdc7e4f74ad1e26666c574fe3" candidate_uuid: "66922775c78f4c65926895f9744e03a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b9be7562599461aac701ceb1544a926"
I20260806 14:58:17.459033 13926 raft_consensus.cc:3037] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 2b9be7562599461aac701ceb1544a926 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:17.460455 14112 raft_consensus.cc:2454] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P c3bf26a2ee61406f9eca9b224c603adc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66922775c78f4c65926895f9744e03a4 in term 1.
I20260806 14:58:17.460455 13926 raft_consensus.cc:2454] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 2b9be7562599461aac701ceb1544a926 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66922775c78f4c65926895f9744e03a4 in term 1.
I20260806 14:58:17.460731 14212 leader_election.cc:302] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66922775c78f4c65926895f9744e03a4, c3bf26a2ee61406f9eca9b224c603adc; no voters:
I20260806 14:58:17.460918 14412 raft_consensus.cc:2781] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:17.461133 14412 raft_consensus.cc:686] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4 [term 1 LEADER]: Becoming Leader. State: Replica: 66922775c78f4c65926895f9744e03a4, State: Running, Role: LEADER
I20260806 14:58:17.461292 14412 consensus_queue.cc:237] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } }
I20260806 14:58:17.462699 13703 catalog_manager.cc:5495] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66922775c78f4c65926895f9744e03a4 (127.11.8.67). New cstate: current_term: 1 leader_uuid: "66922775c78f4c65926895f9744e03a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } health_report { overall_health: UNKNOWN } } }
Aug 06 14:58:17 dist-test-slave-68xk krb5kdc[13639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028296, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:17.541205 14428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.527577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:39323 (local address 127.0.0.1:58338)
0806 14:58:17.527839 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:17.527849 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:17.527981 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:17.528296 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:17.528297 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:17.528874 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:17.528937 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.528940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.530383 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.530385 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.531821 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.531825 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.532707 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.532708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.532800 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.534575 (+ 1775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:17.534593 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:17.537057 (+ 2464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:17.539701 (+ 2644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:17.539709 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:17.539711 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:17.540070 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:17.540385 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:17.540386 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:17.540386 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:17.540482 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:17.540775 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:17.540782 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:17.540842 (+ 60us) client_negotiation.cc:769] Sending connection context
0806 14:58:17.540939 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 14:58:17.541087 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":145,"threads_started":1}
I20260806 14:58:17.541343 13803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.527642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58338 (local address 127.11.8.126:39323)
0806 14:58:17.527768 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:17.527771 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:17.527942 (+ 171us) server_negotiation.cc:407] Connection header received
0806 14:58:17.528154 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:17.528155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:17.528178 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:17.528290 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:17.529097 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.530181 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.531999 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.532414 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.532619 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.537172 (+ 4553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:17.537174 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:17.537174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:17.537182 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:17.539552 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:17.540189 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:17.540191 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:17.540191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:17.540282 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:17.540553 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:17.540553 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:17.540554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:17.540699 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:17.540778 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:17.540965 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 14:58:17.541165 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":83,"server-negotiator.queue_time_us":85}
I20260806 14:58:17.544392 13703 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:58338:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 14:58:17.544804 13703 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 14:58:17.547598 13901 tablet_service.cc:1467] Processing CreateTablet for tablet b1c7e6ae0417445dade80abdddc26b44 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b3c7660880094dea8f8b6c060b08a83a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 14:58:17.547616 14085 tablet_service.cc:1467] Processing CreateTablet for tablet b1c7e6ae0417445dade80abdddc26b44 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b3c7660880094dea8f8b6c060b08a83a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 14:58:17.547657 14270 tablet_service.cc:1467] Processing CreateTablet for tablet b1c7e6ae0417445dade80abdddc26b44 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b3c7660880094dea8f8b6c060b08a83a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 14:58:17.547927 13901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1c7e6ae0417445dade80abdddc26b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:17.547927 14085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1c7e6ae0417445dade80abdddc26b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:17.547941 14270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1c7e6ae0417445dade80abdddc26b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:17.550591 14402 tablet_bootstrap.cc:492] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4: Bootstrap starting.
I20260806 14:58:17.550945 14403 tablet_bootstrap.cc:492] T b1c7e6ae0417445dade80abdddc26b44 P 2b9be7562599461aac701ceb1544a926: Bootstrap starting.
I20260806 14:58:17.551607 14402 tablet_bootstrap.cc:654] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:17.551885 14403 tablet_bootstrap.cc:654] T b1c7e6ae0417445dade80abdddc26b44 P 2b9be7562599461aac701ceb1544a926: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:17.553059 14402 tablet_bootstrap.cc:492] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4: No bootstrap required, opened a new log
I20260806 14:58:17.553049 14404 tablet_bootstrap.cc:492] T b1c7e6ae0417445dade80abdddc26b44 P c3bf26a2ee61406f9eca9b224c603adc: Bootstrap starting.
I20260806 14:58:17.553135 14403 tablet_bootstrap.cc:492] T b1c7e6ae0417445dade80abdddc26b44 P 2b9be7562599461aac701ceb1544a926: No bootstrap required, opened a new log
I20260806 14:58:17.553161 14402 ts_tablet_manager.cc:1397] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 14:58:17.553222 14403 ts_tablet_manager.cc:1397] T b1c7e6ae0417445dade80abdddc26b44 P 2b9be7562599461aac701ceb1544a926: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 14:58:17.553453 14403 raft_consensus.cc:348] T b1c7e6ae0417445dade80abdddc26b44 P 2b9be7562599461aac701ceb1544a926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:17.553489 14402 raft_consensus.cc:348] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:17.553685 14403 raft_consensus.cc:374] T b1c7e6ae0417445dade80abdddc26b44 P 2b9be7562599461aac701ceb1544a926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:17.553709 14402 raft_consensus.cc:374] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:17.553759 14403 raft_consensus.cc:729] T b1c7e6ae0417445dade80abdddc26b44 P 2b9be7562599461aac701ceb1544a926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b9be7562599461aac701ceb1544a926, State: Initialized, Role: FOLLOWER
I20260806 14:58:17.553784 14402 raft_consensus.cc:729] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66922775c78f4c65926895f9744e03a4, State: Initialized, Role: FOLLOWER
I20260806 14:58:17.553869 14404 tablet_bootstrap.cc:654] T b1c7e6ae0417445dade80abdddc26b44 P c3bf26a2ee61406f9eca9b224c603adc: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:17.553889 14402 consensus_queue.cc:260] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:17.553890 14403 consensus_queue.cc:260] T b1c7e6ae0417445dade80abdddc26b44 P 2b9be7562599461aac701ceb1544a926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:17.554113 14403 ts_tablet_manager.cc:1428] T b1c7e6ae0417445dade80abdddc26b44 P 2b9be7562599461aac701ceb1544a926: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:17.554275 14413 tablet_replica.cc:439] T b1c7e6ae0417445dade80abdddc26b44 P 2b9be7562599461aac701ceb1544a926: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } } }
I20260806 14:58:17.554517 14413 tablet_replica.cc:442] T b1c7e6ae0417445dade80abdddc26b44 P 2b9be7562599461aac701ceb1544a926: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 14:58:17.554771 14402 ts_tablet_manager.cc:1428] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260806 14:58:17.554884 14412 tablet_replica.cc:439] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } } }
I20260806 14:58:17.555081 14412 tablet_replica.cc:442] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 14:58:17.555493 14404 tablet_bootstrap.cc:492] T b1c7e6ae0417445dade80abdddc26b44 P c3bf26a2ee61406f9eca9b224c603adc: No bootstrap required, opened a new log
I20260806 14:58:17.555646 14404 ts_tablet_manager.cc:1397] T b1c7e6ae0417445dade80abdddc26b44 P c3bf26a2ee61406f9eca9b224c603adc: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 14:58:17.555925 14404 raft_consensus.cc:348] T b1c7e6ae0417445dade80abdddc26b44 P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:17.556061 14404 raft_consensus.cc:374] T b1c7e6ae0417445dade80abdddc26b44 P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:17.556124 14404 raft_consensus.cc:729] T b1c7e6ae0417445dade80abdddc26b44 P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3bf26a2ee61406f9eca9b224c603adc, State: Initialized, Role: FOLLOWER
I20260806 14:58:17.556244 14404 consensus_queue.cc:260] T b1c7e6ae0417445dade80abdddc26b44 P c3bf26a2ee61406f9eca9b224c603adc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:17.556466 14404 ts_tablet_manager.cc:1428] T b1c7e6ae0417445dade80abdddc26b44 P c3bf26a2ee61406f9eca9b224c603adc: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 14:58:17.556696 14414 tablet_replica.cc:439] T b1c7e6ae0417445dade80abdddc26b44 P c3bf26a2ee61406f9eca9b224c603adc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } } }
I20260806 14:58:17.556833 14414 tablet_replica.cc:442] T b1c7e6ae0417445dade80abdddc26b44 P c3bf26a2ee61406f9eca9b224c603adc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 14:58:17.566331 14412 raft_consensus.cc:482] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 14:58:17.566507 14412 raft_consensus.cc:504] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:17.566710 14412 leader_election.cc:288] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b9be7562599461aac701ceb1544a926 (127.11.8.65:36799), c3bf26a2ee61406f9eca9b224c603adc (127.11.8.66:41741)
I20260806 14:58:17.566941 14112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1c7e6ae0417445dade80abdddc26b44" candidate_uuid: "66922775c78f4c65926895f9744e03a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3bf26a2ee61406f9eca9b224c603adc" is_pre_election: true
I20260806 14:58:17.567028 14112 raft_consensus.cc:2454] T b1c7e6ae0417445dade80abdddc26b44 P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66922775c78f4c65926895f9744e03a4 in term 0.
I20260806 14:58:17.566968 13926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1c7e6ae0417445dade80abdddc26b44" candidate_uuid: "66922775c78f4c65926895f9744e03a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b9be7562599461aac701ceb1544a926" is_pre_election: true
I20260806 14:58:17.567144 13926 raft_consensus.cc:2454] T b1c7e6ae0417445dade80abdddc26b44 P 2b9be7562599461aac701ceb1544a926 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66922775c78f4c65926895f9744e03a4 in term 0.
I20260806 14:58:17.567257 14212 leader_election.cc:302] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66922775c78f4c65926895f9744e03a4, c3bf26a2ee61406f9eca9b224c603adc; no voters:
I20260806 14:58:17.567479 14412 raft_consensus.cc:2781] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 14:58:17.567528 14412 raft_consensus.cc:482] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 14:58:17.567553 14412 raft_consensus.cc:3037] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:17.568454 14412 raft_consensus.cc:504] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:17.568650 14412 leader_election.cc:288] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4 [CANDIDATE]: Term 1 election: Requested vote from peers 2b9be7562599461aac701ceb1544a926 (127.11.8.65:36799), c3bf26a2ee61406f9eca9b224c603adc (127.11.8.66:41741)
I20260806 14:58:17.568917 14112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1c7e6ae0417445dade80abdddc26b44" candidate_uuid: "66922775c78f4c65926895f9744e03a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3bf26a2ee61406f9eca9b224c603adc"
I20260806 14:58:17.569034 14112 raft_consensus.cc:3037] T b1c7e6ae0417445dade80abdddc26b44 P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:17.570232 14112 raft_consensus.cc:2454] T b1c7e6ae0417445dade80abdddc26b44 P c3bf26a2ee61406f9eca9b224c603adc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66922775c78f4c65926895f9744e03a4 in term 1.
I20260806 14:58:17.570464 13926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1c7e6ae0417445dade80abdddc26b44" candidate_uuid: "66922775c78f4c65926895f9744e03a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b9be7562599461aac701ceb1544a926"
I20260806 14:58:17.570533 14212 leader_election.cc:302] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66922775c78f4c65926895f9744e03a4, c3bf26a2ee61406f9eca9b224c603adc; no voters:
I20260806 14:58:17.570587 13926 raft_consensus.cc:3037] T b1c7e6ae0417445dade80abdddc26b44 P 2b9be7562599461aac701ceb1544a926 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:17.570708 14412 raft_consensus.cc:2781] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:17.570792 14412 raft_consensus.cc:686] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4 [term 1 LEADER]: Becoming Leader. State: Replica: 66922775c78f4c65926895f9744e03a4, State: Running, Role: LEADER
I20260806 14:58:17.570871 14412 consensus_queue.cc:237] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:17.571403 14412 tablet_replica.cc:439] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4: TxnStatusTablet state changed. Reason: New leader 66922775c78f4c65926895f9744e03a4. Latest consensus state: current_term: 1 leader_uuid: "66922775c78f4c65926895f9744e03a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } } }
I20260806 14:58:17.571527 14412 tablet_replica.cc:442] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4: This TxnStatusTablet replica's current role is: LEADER
I20260806 14:58:17.572026 13926 raft_consensus.cc:2454] T b1c7e6ae0417445dade80abdddc26b44 P 2b9be7562599461aac701ceb1544a926 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66922775c78f4c65926895f9744e03a4 in term 1.
I20260806 14:58:17.572122 14433 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 14:58:17.572353 14433 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 14:58:17.572368 13702 catalog_manager.cc:5495] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66922775c78f4c65926895f9744e03a4 (127.11.8.67). New cstate: current_term: 1 leader_uuid: "66922775c78f4c65926895f9744e03a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } health_report { overall_health: HEALTHY } } }
I20260806 14:58:17.582538 14399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.578263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44218 (local address 127.11.8.67:40001)
0806 14:58:17.578342 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:17.578344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:17.578488 (+ 144us) server_negotiation.cc:407] Connection header received
0806 14:58:17.578640 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:17.578641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:17.578674 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:17.578780 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 14:58:17.579046 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.579800 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.581331 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.581619 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.581718 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.582023 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 14:58:17.582174 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 14:58:17.582276 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:17.582365 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 14:58:17.582411 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 14:58:17.582626 14428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.578195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:40001 (local address 127.0.0.1:44218)
0806 14:58:17.578344 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:17.578356 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:17.578527 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:17.578804 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:17.578805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:17.578806 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 14:58:17.578912 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.578915 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.579919 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.579920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.581201 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.581205 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.581734 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.581735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.581822 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.581858 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 14:58:17.582261 (+ 403us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 14:58:17.582278 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 14:58:17.582359 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 14:58:17.582468 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260806 14:58:17.585680 13926 raft_consensus.cc:1261] T b1c7e6ae0417445dade80abdddc26b44 P 2b9be7562599461aac701ceb1544a926 [term 1 FOLLOWER]: Refusing update from remote peer 66922775c78f4c65926895f9744e03a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:17.585696 14112 raft_consensus.cc:1261] T b1c7e6ae0417445dade80abdddc26b44 P c3bf26a2ee61406f9eca9b224c603adc [term 1 FOLLOWER]: Refusing update from remote peer 66922775c78f4c65926895f9744e03a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:17.586299 14416 consensus_queue.cc:1035] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:17.586539 14412 consensus_queue.cc:1035] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:17.588721 14414 tablet_replica.cc:439] T b1c7e6ae0417445dade80abdddc26b44 P c3bf26a2ee61406f9eca9b224c603adc: TxnStatusTablet state changed. Reason: New leader 66922775c78f4c65926895f9744e03a4. Latest consensus state: current_term: 1 leader_uuid: "66922775c78f4c65926895f9744e03a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } } }
I20260806 14:58:17.588877 14414 tablet_replica.cc:442] T b1c7e6ae0417445dade80abdddc26b44 P c3bf26a2ee61406f9eca9b224c603adc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 14:58:17.589607 14413 tablet_replica.cc:439] T b1c7e6ae0417445dade80abdddc26b44 P 2b9be7562599461aac701ceb1544a926: TxnStatusTablet state changed. Reason: New leader 66922775c78f4c65926895f9744e03a4. Latest consensus state: current_term: 1 leader_uuid: "66922775c78f4c65926895f9744e03a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } } }
I20260806 14:58:17.589766 14413 tablet_replica.cc:442] T b1c7e6ae0417445dade80abdddc26b44 P 2b9be7562599461aac701ceb1544a926: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 14:58:17.591118 14412 tablet_replica.cc:439] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "66922775c78f4c65926895f9744e03a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } } }
I20260806 14:58:17.591265 14412 tablet_replica.cc:442] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4: This TxnStatusTablet replica's current role is: LEADER
I20260806 14:58:17.591431 14414 tablet_replica.cc:439] T b1c7e6ae0417445dade80abdddc26b44 P c3bf26a2ee61406f9eca9b224c603adc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "66922775c78f4c65926895f9744e03a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } } }
I20260806 14:58:17.591589 14414 tablet_replica.cc:442] T b1c7e6ae0417445dade80abdddc26b44 P c3bf26a2ee61406f9eca9b224c603adc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 14:58:17.591590 14440 mvcc.cc:205] Tried to move back new op lower bound from 7315571906906873856 to 7315571906850807808. Current Snapshot: MvccSnapshot[applied={T|T < 7315571906906873856}]
I20260806 14:58:17.592093 14416 tablet_replica.cc:439] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "66922775c78f4c65926895f9744e03a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } } }
I20260806 14:58:17.592226 14416 tablet_replica.cc:442] T b1c7e6ae0417445dade80abdddc26b44 P 66922775c78f4c65926895f9744e03a4: This TxnStatusTablet replica's current role is: LEADER
I20260806 14:58:17.592849 14441 mvcc.cc:205] Tried to move back new op lower bound from 7315571906906873856 to 7315571906850807808. Current Snapshot: MvccSnapshot[applied={T|T < 7315571906906873856}]
I20260806 14:58:17.593623 14413 tablet_replica.cc:439] T b1c7e6ae0417445dade80abdddc26b44 P 2b9be7562599461aac701ceb1544a926: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "66922775c78f4c65926895f9744e03a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } } }
I20260806 14:58:17.593771 14413 tablet_replica.cc:442] T b1c7e6ae0417445dade80abdddc26b44 P 2b9be7562599461aac701ceb1544a926: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 14:58:17.593607 14442 mvcc.cc:205] Tried to move back new op lower bound from 7315571906906873856 to 7315571906850807808. Current Snapshot: MvccSnapshot[applied={T|T < 7315571906906873856}]
I20260806 14:58:17.597991 14399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.594910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44222 (local address 127.11.8.67:40001)
0806 14:58:17.595016 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:17.595018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:17.595033 (+ 15us) server_negotiation.cc:407] Connection header received
0806 14:58:17.595064 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:17.595065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:17.595090 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:17.595201 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 14:58:17.595401 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.596015 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.597031 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.597282 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.597394 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.597632 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 14:58:17.597679 (+ 47us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 14:58:17.597799 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:17.597862 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 14:58:17.597898 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 14:58:17.605010 14399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.601216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:52001 (local address 127.11.8.67:40001)
0806 14:58:17.601272 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:17.601273 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:17.601411 (+ 138us) server_negotiation.cc:407] Connection header received
0806 14:58:17.601522 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:17.601523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:17.601531 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:17.601604 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 14:58:17.601801 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.602329 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.603968 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.604232 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.604337 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.604663 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 14:58:17.604706 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 14:58:17.604807 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:17.604930 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 14:58:17.604946 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 14:58:17.605144 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.601187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:40001 (local address 127.11.8.67:52001)
0806 14:58:17.601320 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:17.601332 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:17.601415 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:17.601615 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:17.601616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:17.601617 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 14:58:17.601713 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.601715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.602445 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.602446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.603845 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.603848 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.604358 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.604359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.604480 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.604504 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 14:58:17.604825 (+ 321us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 14:58:17.604827 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 14:58:17.604918 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 14:58:17.604988 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 14:58:17.608143 14112 raft_consensus.cc:1261] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P c3bf26a2ee61406f9eca9b224c603adc [term 1 FOLLOWER]: Refusing update from remote peer 66922775c78f4c65926895f9744e03a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:17.608266 13926 raft_consensus.cc:1261] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 2b9be7562599461aac701ceb1544a926 [term 1 FOLLOWER]: Refusing update from remote peer 66922775c78f4c65926895f9744e03a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:17.608475 14452 consensus_queue.cc:1035] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:17.608702 14416 consensus_queue.cc:1035] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:17.628701 13702 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:58:17.628867 13702 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:58:17.629978 13702 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 09dc9af5f18845a3aab211a08457af34: Sending DeleteTablet for 3 replicas of tablet fdb2e9ccdc7e4f74ad1e26666c574fe3
I20260806 14:58:17.630540 13901 tablet_service.cc:1514] Processing DeleteTablet for tablet fdb2e9ccdc7e4f74ad1e26666c574fe3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:17 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:40680
I20260806 14:58:17.630561 14085 tablet_service.cc:1514] Processing DeleteTablet for tablet fdb2e9ccdc7e4f74ad1e26666c574fe3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:17 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:60438
I20260806 14:58:17.630959 14258 tablet_service.cc:1514] Processing DeleteTablet for tablet fdb2e9ccdc7e4f74ad1e26666c574fe3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:17 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:44214
I20260806 14:58:17.631268 14459 tablet_replica.cc:330] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 2b9be7562599461aac701ceb1544a926: stopping tablet replica
I20260806 14:58:17.631443 14461 tablet_replica.cc:330] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4: stopping tablet replica
I20260806 14:58:17.631487 14459 raft_consensus.cc:2229] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 2b9be7562599461aac701ceb1544a926 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:58:17.631623 14459 raft_consensus.cc:2258] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 2b9be7562599461aac701ceb1544a926 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:17.631796 14461 raft_consensus.cc:2229] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 14:58:17.631732 13702 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 14:58:17.631956 13702 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 14:58:17.632285 14461 raft_consensus.cc:2258] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:17.632844 14460 tablet_replica.cc:330] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P c3bf26a2ee61406f9eca9b224c603adc: stopping tablet replica
I20260806 14:58:17.632993 14460 raft_consensus.cc:2229] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P c3bf26a2ee61406f9eca9b224c603adc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:58:17.633102 14460 raft_consensus.cc:2258] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P c3bf26a2ee61406f9eca9b224c603adc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:17.633437 14461 ts_tablet_manager.cc:1905] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:17.633905 14460 ts_tablet_manager.cc:1905] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P c3bf26a2ee61406f9eca9b224c603adc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:17.634629 14085 tablet_service.cc:1467] Processing CreateTablet for tablet ccae508fa1d240afbfa14709bd63369e (DEFAULT_TABLE table=test-table [id=24e3a8fa9d774f57a9e570f73297e09b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:17.634650 13901 tablet_service.cc:1467] Processing CreateTablet for tablet ccae508fa1d240afbfa14709bd63369e (DEFAULT_TABLE table=test-table [id=24e3a8fa9d774f57a9e570f73297e09b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:17.634827 14085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccae508fa1d240afbfa14709bd63369e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:17.634835 13901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccae508fa1d240afbfa14709bd63369e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:17.635730 14459 ts_tablet_manager.cc:1905] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 2b9be7562599461aac701ceb1544a926: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:17.636626 14461 ts_tablet_manager.cc:1918] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 14:58:17.636802 14461 log.cc:1196] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-2/wal/wals/fdb2e9ccdc7e4f74ad1e26666c574fe3
I20260806 14:58:17.637048 14258 tablet_service.cc:1467] Processing CreateTablet for tablet ccae508fa1d240afbfa14709bd63369e (DEFAULT_TABLE table=test-table [id=24e3a8fa9d774f57a9e570f73297e09b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:17.637255 14258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccae508fa1d240afbfa14709bd63369e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:17.637322 14403 tablet_bootstrap.cc:492] T ccae508fa1d240afbfa14709bd63369e P 2b9be7562599461aac701ceb1544a926: Bootstrap starting.
I20260806 14:58:17.637669 14461 ts_tablet_manager.cc:1939] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 66922775c78f4c65926895f9744e03a4: Deleting consensus metadata
I20260806 14:58:17.638034 14403 tablet_bootstrap.cc:654] T ccae508fa1d240afbfa14709bd63369e P 2b9be7562599461aac701ceb1544a926: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:17.638492 13687 catalog_manager.cc:4841] TS 66922775c78f4c65926895f9744e03a4 (127.11.8.67:40001): tablet fdb2e9ccdc7e4f74ad1e26666c574fe3 (table test-table [id=529e8c0061f74a76bcc2e54d6360bf4c]) successfully deleted
I20260806 14:58:17.638779 14459 ts_tablet_manager.cc:1918] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 2b9be7562599461aac701ceb1544a926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 14:58:17.638890 14459 log.cc:1196] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 2b9be7562599461aac701ceb1544a926: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-0/wal/wals/fdb2e9ccdc7e4f74ad1e26666c574fe3
I20260806 14:58:17.639344 14459 ts_tablet_manager.cc:1939] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P 2b9be7562599461aac701ceb1544a926: Deleting consensus metadata
I20260806 14:58:17.639509 14404 tablet_bootstrap.cc:492] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc: Bootstrap starting.
I20260806 14:58:17.640097 13686 catalog_manager.cc:4841] TS 2b9be7562599461aac701ceb1544a926 (127.11.8.65:36799): tablet fdb2e9ccdc7e4f74ad1e26666c574fe3 (table test-table [id=529e8c0061f74a76bcc2e54d6360bf4c]) successfully deleted
I20260806 14:58:17.640168 14404 tablet_bootstrap.cc:654] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:17.640599 14402 tablet_bootstrap.cc:492] T ccae508fa1d240afbfa14709bd63369e P 66922775c78f4c65926895f9744e03a4: Bootstrap starting.
I20260806 14:58:17.641079 14404 tablet_bootstrap.cc:492] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc: No bootstrap required, opened a new log
I20260806 14:58:17.641165 14404 ts_tablet_manager.cc:1397] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 14:58:17.641233 14402 tablet_bootstrap.cc:654] T ccae508fa1d240afbfa14709bd63369e P 66922775c78f4c65926895f9744e03a4: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:17.641371 14404 raft_consensus.cc:348] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:17.641479 14404 raft_consensus.cc:374] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:17.641516 14404 raft_consensus.cc:729] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3bf26a2ee61406f9eca9b224c603adc, State: Initialized, Role: FOLLOWER
I20260806 14:58:17.641602 14404 consensus_queue.cc:260] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:17.641753 14404 ts_tablet_manager.cc:1428] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 14:58:17.642583 14402 tablet_bootstrap.cc:492] T ccae508fa1d240afbfa14709bd63369e P 66922775c78f4c65926895f9744e03a4: No bootstrap required, opened a new log
I20260806 14:58:17.642668 14402 ts_tablet_manager.cc:1397] T ccae508fa1d240afbfa14709bd63369e P 66922775c78f4c65926895f9744e03a4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 14:58:17.642937 14402 raft_consensus.cc:348] T ccae508fa1d240afbfa14709bd63369e P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:17.643064 14402 raft_consensus.cc:374] T ccae508fa1d240afbfa14709bd63369e P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:17.643105 14402 raft_consensus.cc:729] T ccae508fa1d240afbfa14709bd63369e P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66922775c78f4c65926895f9744e03a4, State: Initialized, Role: FOLLOWER
I20260806 14:58:17.643210 14402 consensus_queue.cc:260] T ccae508fa1d240afbfa14709bd63369e P 66922775c78f4c65926895f9744e03a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:17.643353 14402 ts_tablet_manager.cc:1428] T ccae508fa1d240afbfa14709bd63369e P 66922775c78f4c65926895f9744e03a4: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 14:58:17.644656 14460 ts_tablet_manager.cc:1918] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P c3bf26a2ee61406f9eca9b224c603adc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 14:58:17.644752 14460 log.cc:1196] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P c3bf26a2ee61406f9eca9b224c603adc: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-1/wal/wals/fdb2e9ccdc7e4f74ad1e26666c574fe3
I20260806 14:58:17.645069 14460 ts_tablet_manager.cc:1939] T fdb2e9ccdc7e4f74ad1e26666c574fe3 P c3bf26a2ee61406f9eca9b224c603adc: Deleting consensus metadata
I20260806 14:58:17.645577 13685 catalog_manager.cc:4841] TS c3bf26a2ee61406f9eca9b224c603adc (127.11.8.66:41741): tablet fdb2e9ccdc7e4f74ad1e26666c574fe3 (table test-table [id=529e8c0061f74a76bcc2e54d6360bf4c]) successfully deleted
I20260806 14:58:17.647096 14403 tablet_bootstrap.cc:492] T ccae508fa1d240afbfa14709bd63369e P 2b9be7562599461aac701ceb1544a926: No bootstrap required, opened a new log
I20260806 14:58:17.647184 14403 ts_tablet_manager.cc:1397] T ccae508fa1d240afbfa14709bd63369e P 2b9be7562599461aac701ceb1544a926: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20260806 14:58:17.647410 14403 raft_consensus.cc:348] T ccae508fa1d240afbfa14709bd63369e P 2b9be7562599461aac701ceb1544a926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:17.647536 14403 raft_consensus.cc:374] T ccae508fa1d240afbfa14709bd63369e P 2b9be7562599461aac701ceb1544a926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:17.647578 14403 raft_consensus.cc:729] T ccae508fa1d240afbfa14709bd63369e P 2b9be7562599461aac701ceb1544a926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b9be7562599461aac701ceb1544a926, State: Initialized, Role: FOLLOWER
I20260806 14:58:17.647681 14403 consensus_queue.cc:260] T ccae508fa1d240afbfa14709bd63369e P 2b9be7562599461aac701ceb1544a926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:17.647833 14403 ts_tablet_manager.cc:1428] T ccae508fa1d240afbfa14709bd63369e P 2b9be7562599461aac701ceb1544a926: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:17.683137 14414 raft_consensus.cc:482] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 14:58:17.683250 14414 raft_consensus.cc:504] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:17.683615 14414 leader_election.cc:288] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b9be7562599461aac701ceb1544a926 (127.11.8.65:36799), 66922775c78f4c65926895f9744e03a4 (127.11.8.67:40001)
I20260806 14:58:17.686431 14413 raft_consensus.cc:482] T ccae508fa1d240afbfa14709bd63369e P 2b9be7562599461aac701ceb1544a926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 14:58:17.686517 14413 raft_consensus.cc:504] T ccae508fa1d240afbfa14709bd63369e P 2b9be7562599461aac701ceb1544a926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:17.686825 14413 leader_election.cc:288] T ccae508fa1d240afbfa14709bd63369e P 2b9be7562599461aac701ceb1544a926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3bf26a2ee61406f9eca9b224c603adc (127.11.8.66:41741), 66922775c78f4c65926895f9744e03a4 (127.11.8.67:40001)
I20260806 14:58:17.687597 14399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.683834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:35179 (local address 127.11.8.67:40001)
0806 14:58:17.683894 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:17.683896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:17.684036 (+ 140us) server_negotiation.cc:407] Connection header received
0806 14:58:17.684132 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:17.684133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:17.684144 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:17.684203 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:17.684399 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.685127 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.686606 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.687172 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.687216 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.687229 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:17.687473 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 14:58:17.687518 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 14:58:17.687878 14466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.683788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:40001 (local address 127.11.8.66:35179)
0806 14:58:17.683958 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:17.683968 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:17.684066 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:17.684217 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:17.684218 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:17.684220 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:17.684287 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.684289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.685212 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.685212 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.686500 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.686503 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.687296 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.687297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.687383 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.687388 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 14:58:17.687742 (+ 354us) client_negotiation.cc:240] Negotiation successful
0806 14:58:17.687786 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":6,"thread_start_us":70,"threads_started":1}
I20260806 14:58:17.688277 14306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccae508fa1d240afbfa14709bd63369e" candidate_uuid: "c3bf26a2ee61406f9eca9b224c603adc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66922775c78f4c65926895f9744e03a4" is_pre_election: true
I20260806 14:58:17.688413 14306 raft_consensus.cc:2454] T ccae508fa1d240afbfa14709bd63369e P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3bf26a2ee61406f9eca9b224c603adc in term 0.
I20260806 14:58:17.688689 14026 leader_election.cc:302] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66922775c78f4c65926895f9744e03a4, c3bf26a2ee61406f9eca9b224c603adc; no voters:
I20260806 14:58:17.688815 14414 raft_consensus.cc:2781] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 14:58:17.688851 14414 raft_consensus.cc:482] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 14:58:17.688871 14414 raft_consensus.cc:3037] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:17.689797 14414 raft_consensus.cc:504] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:17.689918 14414 leader_election.cc:288] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc [CANDIDATE]: Term 1 election: Requested vote from peers 2b9be7562599461aac701ceb1544a926 (127.11.8.65:36799), 66922775c78f4c65926895f9744e03a4 (127.11.8.67:40001)
I20260806 14:58:17.690231 14306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccae508fa1d240afbfa14709bd63369e" candidate_uuid: "c3bf26a2ee61406f9eca9b224c603adc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66922775c78f4c65926895f9744e03a4"
I20260806 14:58:17.690310 14306 raft_consensus.cc:3037] T ccae508fa1d240afbfa14709bd63369e P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:17.691152 14306 raft_consensus.cc:2454] T ccae508fa1d240afbfa14709bd63369e P 66922775c78f4c65926895f9744e03a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3bf26a2ee61406f9eca9b224c603adc in term 1.
I20260806 14:58:17.691226 14399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.687473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:39121 (local address 127.11.8.67:40001)
0806 14:58:17.687678 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:17.687680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:17.688079 (+ 399us) server_negotiation.cc:407] Connection header received
0806 14:58:17.688174 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:17.688175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:17.688184 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:17.688212 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:17.688413 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.689037 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.690306 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.690824 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.690873 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.690883 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:17.691129 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 14:58:17.691165 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":62,"threads_started":1}
I20260806 14:58:17.691291 14400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.687478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:44441 (local address 127.11.8.66:41741)
0806 14:58:17.687572 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:17.687574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:17.687675 (+ 101us) server_negotiation.cc:407] Connection header received
0806 14:58:17.687781 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:17.687782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:17.687792 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:17.687821 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:17.688035 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.688727 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.690303 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.690892 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.690937 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.690946 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:17.691190 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 14:58:17.691229 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 14:58:17.691470 14468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.687192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:40001 (local address 127.11.8.65:39121)
0806 14:58:17.687992 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:17.688004 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:17.688106 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:17.688270 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:17.688271 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:17.688273 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:17.688335 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.688336 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.689136 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.689137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.690210 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.690214 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.690948 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.690948 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.691036 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.691040 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 14:58:17.691331 (+ 291us) client_negotiation.cc:240] Negotiation successful
0806 14:58:17.691379 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":785,"thread_start_us":157,"threads_started":1}
I20260806 14:58:17.691620 14026 leader_election.cc:302] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66922775c78f4c65926895f9744e03a4, c3bf26a2ee61406f9eca9b224c603adc; no voters:
I20260806 14:58:17.691673 14470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.687016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:41741 (local address 127.11.8.65:44441)
0806 14:58:17.687544 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:17.687555 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:17.687706 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:17.687879 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:17.687880 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:17.687882 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:17.687951 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.687953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.688886 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.688887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.690201 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.690204 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.691009 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.691009 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.691099 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.691103 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 14:58:17.691397 (+ 294us) client_negotiation.cc:240] Negotiation successful
0806 14:58:17.691438 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"mutex_wait_us":39,"thread_start_us":51,"threads_started":1}
I20260806 14:58:17.691784 14306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccae508fa1d240afbfa14709bd63369e" candidate_uuid: "2b9be7562599461aac701ceb1544a926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66922775c78f4c65926895f9744e03a4" is_pre_election: true
I20260806 14:58:17.691875 14306 raft_consensus.cc:2379] T ccae508fa1d240afbfa14709bd63369e P 66922775c78f4c65926895f9744e03a4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2b9be7562599461aac701ceb1544a926 in current term 1: Already voted for candidate c3bf26a2ee61406f9eca9b224c603adc in this term.
I20260806 14:58:17.691910 14414 raft_consensus.cc:2781] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:17.691983 14112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccae508fa1d240afbfa14709bd63369e" candidate_uuid: "2b9be7562599461aac701ceb1544a926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3bf26a2ee61406f9eca9b224c603adc" is_pre_election: true
I20260806 14:58:17.692070 14414 raft_consensus.cc:686] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc [term 1 LEADER]: Becoming Leader. State: Replica: c3bf26a2ee61406f9eca9b224c603adc, State: Running, Role: LEADER
I20260806 14:58:17.692145 14414 consensus_queue.cc:237] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:17.692617 13834 leader_election.cc:302] T ccae508fa1d240afbfa14709bd63369e P 2b9be7562599461aac701ceb1544a926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2b9be7562599461aac701ceb1544a926; no voters: 66922775c78f4c65926895f9744e03a4, c3bf26a2ee61406f9eca9b224c603adc
I20260806 14:58:17.692799 14413 raft_consensus.cc:3037] T ccae508fa1d240afbfa14709bd63369e P 2b9be7562599461aac701ceb1544a926 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:17.693030 14467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.683804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:36799 (local address 127.11.8.66:55621)
0806 14:58:17.684020 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:17.684030 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:17.684073 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:17.684372 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:17.684373 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:17.684375 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:17.684437 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.684439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.685091 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.685092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.686605 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.686607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.687604 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.687605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.687871 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.687876 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 14:58:17.692889 (+ 5013us) client_negotiation.cc:240] Negotiation successful
0806 14:58:17.692943 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":82,"thread_start_us":105,"threads_started":1}
I20260806 14:58:17.693183 13702 catalog_manager.cc:5495] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc reported cstate change: term changed from 0 to 1, leader changed from <none> to c3bf26a2ee61406f9eca9b224c603adc (127.11.8.66). New cstate: current_term: 1 leader_uuid: "c3bf26a2ee61406f9eca9b224c603adc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } health_report { overall_health: UNKNOWN } } }
I20260806 14:58:17.693689 14413 raft_consensus.cc:2726] T ccae508fa1d240afbfa14709bd63369e P 2b9be7562599461aac701ceb1544a926 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 14:58:17.693975 14401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.684020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:55621 (local address 127.11.8.65:36799)
0806 14:58:17.684063 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:17.684065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:17.684075 (+ 10us) server_negotiation.cc:407] Connection header received
0806 14:58:17.684260 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:17.684261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:17.684269 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:17.684340 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:17.684504 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.684972 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.686746 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.687299 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.687338 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.687346 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:17.692722 (+ 5376us) server_negotiation.cc:299] Negotiation successful
0806 14:58:17.692773 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 14:58:17.694196 13926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccae508fa1d240afbfa14709bd63369e" candidate_uuid: "c3bf26a2ee61406f9eca9b224c603adc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b9be7562599461aac701ceb1544a926" is_pre_election: true
I20260806 14:58:17.694290 13926 raft_consensus.cc:2454] T ccae508fa1d240afbfa14709bd63369e P 2b9be7562599461aac701ceb1544a926 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3bf26a2ee61406f9eca9b224c603adc in term 1.
I20260806 14:58:17.694373 13925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccae508fa1d240afbfa14709bd63369e" candidate_uuid: "c3bf26a2ee61406f9eca9b224c603adc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b9be7562599461aac701ceb1544a926"
I20260806 14:58:17.695235 13925 raft_consensus.cc:2454] T ccae508fa1d240afbfa14709bd63369e P 2b9be7562599461aac701ceb1544a926 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3bf26a2ee61406f9eca9b224c603adc in term 1.
I20260806 14:58:17.714437 14400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.711920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60440 (local address 127.11.8.66:41741)
0806 14:58:17.711968 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:17.711971 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:17.711997 (+ 26us) server_negotiation.cc:407] Connection header received
0806 14:58:17.712036 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:17.712037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:17.712058 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:17.712143 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 14:58:17.712272 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.712878 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.713616 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.713830 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.713910 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.714059 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 14:58:17.714169 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 14:58:17.714248 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:17.714311 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 14:58:17.714331 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 14:58:17.719920 14483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.717167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:40001 (local address 127.11.8.66:38387)
0806 14:58:17.717333 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:17.717344 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:17.717436 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:17.717606 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:17.717607 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:17.717608 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 14:58:17.717680 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.717682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.718461 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.718462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.719061 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.719063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.719490 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.719492 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.719564 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.719581 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 14:58:17.719743 (+ 162us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 14:58:17.719745 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 14:58:17.719787 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 14:58:17.719825 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
I20260806 14:58:17.719918 14471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.717256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:38387 (local address 127.11.8.67:40001)
0806 14:58:17.717306 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:17.717308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:17.717420 (+ 112us) server_negotiation.cc:407] Connection header received
0806 14:58:17.717500 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:17.717501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:17.717511 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:17.717604 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 14:58:17.717747 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.718351 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.719154 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.719419 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.719477 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.719656 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 14:58:17.719689 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 14:58:17.719732 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:17.719788 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 14:58:17.719810 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 14:58:17.722503 14306 raft_consensus.cc:1261] T ccae508fa1d240afbfa14709bd63369e P 66922775c78f4c65926895f9744e03a4 [term 1 FOLLOWER]: Refusing update from remote peer c3bf26a2ee61406f9eca9b224c603adc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:17.722517 13925 raft_consensus.cc:1261] T ccae508fa1d240afbfa14709bd63369e P 2b9be7562599461aac701ceb1544a926 [term 1 FOLLOWER]: Refusing update from remote peer c3bf26a2ee61406f9eca9b224c603adc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:17.722867 14472 consensus_queue.cc:1035] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc [LEADER]: Connected to new peer: Peer: permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:17.723129 14414 consensus_queue.cc:1035] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:17.730648 14400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.728371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:36855 (local address 127.11.8.66:41741)
0806 14:58:17.728417 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:17.728420 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:17.728427 (+ 7us) server_negotiation.cc:407] Connection header received
0806 14:58:17.728450 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:17.728450 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:17.728461 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:17.728524 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 14:58:17.728659 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.729190 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.729936 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.730146 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.730222 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.730381 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 14:58:17.730441 (+ 60us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 14:58:17.730490 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:17.730568 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 14:58:17.730586 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 14:58:17.730675 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:17.727982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:41741 (local address 127.11.8.67:36855)
0806 14:58:17.728054 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:17.728067 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:17.728110 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:17.728547 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:17.728547 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:17.728549 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 14:58:17.728580 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.728581 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.729282 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.729284 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.729845 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:17.729847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:17.730222 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:17.730223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:17.730283 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:17.730287 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 14:58:17.730515 (+ 228us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 14:58:17.730518 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 14:58:17.730564 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 14:58:17.730606 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 14:58:17.731091 14441 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 14:58:19.258842 13703 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:58:19.258962 13703 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:58:19.259958 13703 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 09dc9af5f18845a3aab211a08457af34: Sending DeleteTablet for 3 replicas of tablet ccae508fa1d240afbfa14709bd63369e
I20260806 14:58:19.260245 14085 tablet_service.cc:1514] Processing DeleteTablet for tablet ccae508fa1d240afbfa14709bd63369e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:19 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:60438
I20260806 14:58:19.260282 14258 tablet_service.cc:1514] Processing DeleteTablet for tablet ccae508fa1d240afbfa14709bd63369e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:19 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:44214
I20260806 14:58:19.260298 13901 tablet_service.cc:1514] Processing DeleteTablet for tablet ccae508fa1d240afbfa14709bd63369e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:19 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:40680
I20260806 14:58:19.262794 14522 tablet_replica.cc:330] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc: stopping tablet replica
I20260806 14:58:19.262876 14522 raft_consensus.cc:2229] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc [term 1 LEADER]: Raft consensus shutting down.
I20260806 14:58:19.262974 14522 raft_consensus.cc:2258] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:19.263087 14523 tablet_replica.cc:330] T ccae508fa1d240afbfa14709bd63369e P 66922775c78f4c65926895f9744e03a4: stopping tablet replica
I20260806 14:58:19.263159 14523 raft_consensus.cc:2229] T ccae508fa1d240afbfa14709bd63369e P 66922775c78f4c65926895f9744e03a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:58:19.263202 14523 raft_consensus.cc:2258] T ccae508fa1d240afbfa14709bd63369e P 66922775c78f4c65926895f9744e03a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:19.263309 14525 tablet_replica.cc:330] T ccae508fa1d240afbfa14709bd63369e P 2b9be7562599461aac701ceb1544a926: stopping tablet replica
I20260806 14:58:19.263379 14525 raft_consensus.cc:2229] T ccae508fa1d240afbfa14709bd63369e P 2b9be7562599461aac701ceb1544a926 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:58:19.263427 14525 raft_consensus.cc:2258] T ccae508fa1d240afbfa14709bd63369e P 2b9be7562599461aac701ceb1544a926 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:19.263572 14522 ts_tablet_manager.cc:1905] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:19.263649 14523 ts_tablet_manager.cc:1905] T ccae508fa1d240afbfa14709bd63369e P 66922775c78f4c65926895f9744e03a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:19.263729 14525 ts_tablet_manager.cc:1905] T ccae508fa1d240afbfa14709bd63369e P 2b9be7562599461aac701ceb1544a926: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:19.265677 14522 ts_tablet_manager.cc:1918] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 14:58:19.265750 14522 log.cc:1196] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-1/wal/wals/ccae508fa1d240afbfa14709bd63369e
I20260806 14:58:19.265911 14523 ts_tablet_manager.cc:1918] T ccae508fa1d240afbfa14709bd63369e P 66922775c78f4c65926895f9744e03a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 14:58:19.265981 14523 log.cc:1196] T ccae508fa1d240afbfa14709bd63369e P 66922775c78f4c65926895f9744e03a4: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-2/wal/wals/ccae508fa1d240afbfa14709bd63369e
I20260806 14:58:19.266029 14522 ts_tablet_manager.cc:1939] T ccae508fa1d240afbfa14709bd63369e P c3bf26a2ee61406f9eca9b224c603adc: Deleting consensus metadata
I20260806 14:58:19.266108 14525 ts_tablet_manager.cc:1918] T ccae508fa1d240afbfa14709bd63369e P 2b9be7562599461aac701ceb1544a926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 14:58:19.266180 14525 log.cc:1196] T ccae508fa1d240afbfa14709bd63369e P 2b9be7562599461aac701ceb1544a926: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-0/wal/wals/ccae508fa1d240afbfa14709bd63369e
I20260806 14:58:19.266212 14523 ts_tablet_manager.cc:1939] T ccae508fa1d240afbfa14709bd63369e P 66922775c78f4c65926895f9744e03a4: Deleting consensus metadata
I20260806 14:58:19.266330 13685 catalog_manager.cc:4841] TS c3bf26a2ee61406f9eca9b224c603adc (127.11.8.66:41741): tablet ccae508fa1d240afbfa14709bd63369e (table test-table [id=24e3a8fa9d774f57a9e570f73297e09b]) successfully deleted
I20260806 14:58:19.266465 14525 ts_tablet_manager.cc:1939] T ccae508fa1d240afbfa14709bd63369e P 2b9be7562599461aac701ceb1544a926: Deleting consensus metadata
I20260806 14:58:19.266563 13687 catalog_manager.cc:4841] TS 66922775c78f4c65926895f9744e03a4 (127.11.8.67:40001): tablet ccae508fa1d240afbfa14709bd63369e (table test-table [id=24e3a8fa9d774f57a9e570f73297e09b]) successfully deleted
I20260806 14:58:19.266798 13686 catalog_manager.cc:4841] TS 2b9be7562599461aac701ceb1544a926 (127.11.8.65:36799): tablet ccae508fa1d240afbfa14709bd63369e (table test-table [id=24e3a8fa9d774f57a9e570f73297e09b]) successfully deleted
Aug 06 14:58:19 dist-test-slave-68xk krb5kdc[13639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028299, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 14:58:19 dist-test-slave-68xk krb5kdc[13639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028299, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:19.290956 14535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:19.278918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58342 (local address 127.11.8.126:39323)
0806 14:58:19.279066 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:19.279069 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:19.279079 (+ 10us) server_negotiation.cc:407] Connection header received
0806 14:58:19.279183 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:19.279184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:19.279196 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:19.279284 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:19.279890 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:19.280432 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:19.281200 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:19.281419 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:19.281498 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:19.288234 (+ 6736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:19.288236 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:19.288236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:19.288253 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:19.289639 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:19.290018 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:19.290019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:19.290020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:19.290086 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:19.290307 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:19.290308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:19.290308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:19.290487 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:19.290602 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:19.290742 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 14:58:19.290839 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":77,"threads_started":1}
I20260806 14:58:19.292682 13703 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58342:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 14:58:19.292872 13703 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 14:58:19.294445 14258 tablet_service.cc:1467] Processing CreateTablet for tablet 3485c69bbae542329c77e9277e5a0f41 (DEFAULT_TABLE table=test-table [id=fe06831a1b904658b75717396957bdcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:19.294404 13901 tablet_service.cc:1467] Processing CreateTablet for tablet 3485c69bbae542329c77e9277e5a0f41 (DEFAULT_TABLE table=test-table [id=fe06831a1b904658b75717396957bdcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:19.294461 14085 tablet_service.cc:1467] Processing CreateTablet for tablet 3485c69bbae542329c77e9277e5a0f41 (DEFAULT_TABLE table=test-table [id=fe06831a1b904658b75717396957bdcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:19.294587 14258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3485c69bbae542329c77e9277e5a0f41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:19.294586 13901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3485c69bbae542329c77e9277e5a0f41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:19.294590 14085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3485c69bbae542329c77e9277e5a0f41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:19.295711 14536 tablet_bootstrap.cc:492] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc: Bootstrap starting.
I20260806 14:58:19.296125 14537 tablet_bootstrap.cc:492] T 3485c69bbae542329c77e9277e5a0f41 P 2b9be7562599461aac701ceb1544a926: Bootstrap starting.
I20260806 14:58:19.296221 14536 tablet_bootstrap.cc:654] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:19.296525 14537 tablet_bootstrap.cc:654] T 3485c69bbae542329c77e9277e5a0f41 P 2b9be7562599461aac701ceb1544a926: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:19.296883 14536 tablet_bootstrap.cc:492] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc: No bootstrap required, opened a new log
I20260806 14:58:19.296898 14538 tablet_bootstrap.cc:492] T 3485c69bbae542329c77e9277e5a0f41 P 66922775c78f4c65926895f9744e03a4: Bootstrap starting.
I20260806 14:58:19.296945 14536 ts_tablet_manager.cc:1397] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:19.297072 14536 raft_consensus.cc:348] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:19.297169 14536 raft_consensus.cc:374] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:19.297197 14536 raft_consensus.cc:729] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3bf26a2ee61406f9eca9b224c603adc, State: Initialized, Role: FOLLOWER
I20260806 14:58:19.297242 14538 tablet_bootstrap.cc:654] T 3485c69bbae542329c77e9277e5a0f41 P 66922775c78f4c65926895f9744e03a4: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:19.297247 14537 tablet_bootstrap.cc:492] T 3485c69bbae542329c77e9277e5a0f41 P 2b9be7562599461aac701ceb1544a926: No bootstrap required, opened a new log
I20260806 14:58:19.297295 14537 ts_tablet_manager.cc:1397] T 3485c69bbae542329c77e9277e5a0f41 P 2b9be7562599461aac701ceb1544a926: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:19.297281 14536 consensus_queue.cc:260] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:19.297420 14537 raft_consensus.cc:348] T 3485c69bbae542329c77e9277e5a0f41 P 2b9be7562599461aac701ceb1544a926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:19.297526 14537 raft_consensus.cc:374] T 3485c69bbae542329c77e9277e5a0f41 P 2b9be7562599461aac701ceb1544a926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:19.297530 14536 ts_tablet_manager.cc:1428] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:19.297554 14537 raft_consensus.cc:729] T 3485c69bbae542329c77e9277e5a0f41 P 2b9be7562599461aac701ceb1544a926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b9be7562599461aac701ceb1544a926, State: Initialized, Role: FOLLOWER
I20260806 14:58:19.297626 14537 consensus_queue.cc:260] T 3485c69bbae542329c77e9277e5a0f41 P 2b9be7562599461aac701ceb1544a926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:19.297821 14537 ts_tablet_manager.cc:1428] T 3485c69bbae542329c77e9277e5a0f41 P 2b9be7562599461aac701ceb1544a926: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 14:58:19.298298 14538 tablet_bootstrap.cc:492] T 3485c69bbae542329c77e9277e5a0f41 P 66922775c78f4c65926895f9744e03a4: No bootstrap required, opened a new log
I20260806 14:58:19.298354 14538 ts_tablet_manager.cc:1397] T 3485c69bbae542329c77e9277e5a0f41 P 66922775c78f4c65926895f9744e03a4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:19.298506 14538 raft_consensus.cc:348] T 3485c69bbae542329c77e9277e5a0f41 P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:19.298621 14538 raft_consensus.cc:374] T 3485c69bbae542329c77e9277e5a0f41 P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:19.298655 14538 raft_consensus.cc:729] T 3485c69bbae542329c77e9277e5a0f41 P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66922775c78f4c65926895f9744e03a4, State: Initialized, Role: FOLLOWER
I20260806 14:58:19.298729 14538 consensus_queue.cc:260] T 3485c69bbae542329c77e9277e5a0f41 P 66922775c78f4c65926895f9744e03a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:19.298851 14538 ts_tablet_manager.cc:1428] T 3485c69bbae542329c77e9277e5a0f41 P 66922775c78f4c65926895f9744e03a4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 14:58:19.525974 14511 raft_consensus.cc:482] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 14:58:19.526090 14511 raft_consensus.cc:504] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:19.526286 14511 leader_election.cc:288] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b9be7562599461aac701ceb1544a926 (127.11.8.65:36799), 66922775c78f4c65926895f9744e03a4 (127.11.8.67:40001)
I20260806 14:58:19.526592 14306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3485c69bbae542329c77e9277e5a0f41" candidate_uuid: "c3bf26a2ee61406f9eca9b224c603adc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66922775c78f4c65926895f9744e03a4" is_pre_election: true
I20260806 14:58:19.526582 13925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3485c69bbae542329c77e9277e5a0f41" candidate_uuid: "c3bf26a2ee61406f9eca9b224c603adc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b9be7562599461aac701ceb1544a926" is_pre_election: true
I20260806 14:58:19.526706 14306 raft_consensus.cc:2454] T 3485c69bbae542329c77e9277e5a0f41 P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3bf26a2ee61406f9eca9b224c603adc in term 0.
I20260806 14:58:19.526703 13925 raft_consensus.cc:2454] T 3485c69bbae542329c77e9277e5a0f41 P 2b9be7562599461aac701ceb1544a926 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3bf26a2ee61406f9eca9b224c603adc in term 0.
I20260806 14:58:19.526882 14024 leader_election.cc:302] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b9be7562599461aac701ceb1544a926, c3bf26a2ee61406f9eca9b224c603adc; no voters:
I20260806 14:58:19.527022 14511 raft_consensus.cc:2781] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 14:58:19.527074 14511 raft_consensus.cc:482] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 14:58:19.527105 14511 raft_consensus.cc:3037] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:19.527861 14511 raft_consensus.cc:504] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:19.528008 14511 leader_election.cc:288] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc [CANDIDATE]: Term 1 election: Requested vote from peers 2b9be7562599461aac701ceb1544a926 (127.11.8.65:36799), 66922775c78f4c65926895f9744e03a4 (127.11.8.67:40001)
I20260806 14:58:19.528228 13925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3485c69bbae542329c77e9277e5a0f41" candidate_uuid: "c3bf26a2ee61406f9eca9b224c603adc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b9be7562599461aac701ceb1544a926"
I20260806 14:58:19.528317 13925 raft_consensus.cc:3037] T 3485c69bbae542329c77e9277e5a0f41 P 2b9be7562599461aac701ceb1544a926 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:19.528273 14306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3485c69bbae542329c77e9277e5a0f41" candidate_uuid: "c3bf26a2ee61406f9eca9b224c603adc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66922775c78f4c65926895f9744e03a4"
I20260806 14:58:19.528355 14306 raft_consensus.cc:3037] T 3485c69bbae542329c77e9277e5a0f41 P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:19.528960 14306 raft_consensus.cc:2454] T 3485c69bbae542329c77e9277e5a0f41 P 66922775c78f4c65926895f9744e03a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3bf26a2ee61406f9eca9b224c603adc in term 1.
I20260806 14:58:19.528970 13925 raft_consensus.cc:2454] T 3485c69bbae542329c77e9277e5a0f41 P 2b9be7562599461aac701ceb1544a926 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3bf26a2ee61406f9eca9b224c603adc in term 1.
I20260806 14:58:19.529175 14026 leader_election.cc:302] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66922775c78f4c65926895f9744e03a4, c3bf26a2ee61406f9eca9b224c603adc; no voters:
I20260806 14:58:19.529289 14511 raft_consensus.cc:2781] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:19.529359 14511 raft_consensus.cc:686] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc [term 1 LEADER]: Becoming Leader. State: Replica: c3bf26a2ee61406f9eca9b224c603adc, State: Running, Role: LEADER
I20260806 14:58:19.529415 14511 consensus_queue.cc:237] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:19.530207 13702 catalog_manager.cc:5495] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc reported cstate change: term changed from 0 to 1, leader changed from <none> to c3bf26a2ee61406f9eca9b224c603adc (127.11.8.66). New cstate: current_term: 1 leader_uuid: "c3bf26a2ee61406f9eca9b224c603adc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } health_report { overall_health: UNKNOWN } } }
I20260806 14:58:19.583597 14546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:19.580965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60452 (local address 127.11.8.66:41741)
0806 14:58:19.581137 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:19.581138 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:19.581145 (+ 7us) server_negotiation.cc:407] Connection header received
0806 14:58:19.581163 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:19.581164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:19.581173 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:19.581251 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 14:58:19.581420 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:19.581955 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:19.582792 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:19.583008 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:19.583087 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:19.583263 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 14:58:19.583308 (+ 45us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 14:58:19.583367 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:19.583465 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 14:58:19.583492 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":119,"threads_started":1}
I20260806 14:58:19.585888 13925 raft_consensus.cc:1261] T 3485c69bbae542329c77e9277e5a0f41 P 2b9be7562599461aac701ceb1544a926 [term 1 FOLLOWER]: Refusing update from remote peer c3bf26a2ee61406f9eca9b224c603adc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:19.585902 14306 raft_consensus.cc:1261] T 3485c69bbae542329c77e9277e5a0f41 P 66922775c78f4c65926895f9744e03a4 [term 1 FOLLOWER]: Refusing update from remote peer c3bf26a2ee61406f9eca9b224c603adc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:19.586163 14515 consensus_queue.cc:1035] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:19.586300 14511 consensus_queue.cc:1035] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc [LEADER]: Connected to new peer: Peer: permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:19.596026 13703 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:58:19.596122 13703 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:58:19.596781 13703 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 09dc9af5f18845a3aab211a08457af34: Sending DeleteTablet for 3 replicas of tablet 3485c69bbae542329c77e9277e5a0f41
I20260806 14:58:19.597052 13901 tablet_service.cc:1514] Processing DeleteTablet for tablet 3485c69bbae542329c77e9277e5a0f41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:19 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:40680
I20260806 14:58:19.597074 14085 tablet_service.cc:1514] Processing DeleteTablet for tablet 3485c69bbae542329c77e9277e5a0f41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:19 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:60438
I20260806 14:58:19.597154 14525 tablet_replica.cc:330] T 3485c69bbae542329c77e9277e5a0f41 P 2b9be7562599461aac701ceb1544a926: stopping tablet replica
I20260806 14:58:19.597168 14522 tablet_replica.cc:330] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc: stopping tablet replica
I20260806 14:58:19.597216 14522 raft_consensus.cc:2229] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc [term 1 LEADER]: Raft consensus shutting down.
I20260806 14:58:19.597208 14525 raft_consensus.cc:2229] T 3485c69bbae542329c77e9277e5a0f41 P 2b9be7562599461aac701ceb1544a926 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:58:19.597270 14525 raft_consensus.cc:2258] T 3485c69bbae542329c77e9277e5a0f41 P 2b9be7562599461aac701ceb1544a926 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:19.597275 14522 raft_consensus.cc:2258] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:19.597438 14258 tablet_service.cc:1514] Processing DeleteTablet for tablet 3485c69bbae542329c77e9277e5a0f41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:19 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:44214
I20260806 14:58:19.597563 14523 tablet_replica.cc:330] T 3485c69bbae542329c77e9277e5a0f41 P 66922775c78f4c65926895f9744e03a4: stopping tablet replica
I20260806 14:58:19.597604 14525 ts_tablet_manager.cc:1905] T 3485c69bbae542329c77e9277e5a0f41 P 2b9be7562599461aac701ceb1544a926: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:19.597630 14523 raft_consensus.cc:2229] T 3485c69bbae542329c77e9277e5a0f41 P 66922775c78f4c65926895f9744e03a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:58:19.597649 14522 ts_tablet_manager.cc:1905] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:19.597668 14523 raft_consensus.cc:2258] T 3485c69bbae542329c77e9277e5a0f41 P 66922775c78f4c65926895f9744e03a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:19.597605 13703 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58342:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 14:58:19.597740 13703 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 14:58:19.599046 14258 tablet_service.cc:1467] Processing CreateTablet for tablet 2c7ddd428afc4d3598f047b4befeb17b (DEFAULT_TABLE table=test-table [id=0936eae5a27d470ab920901b8d45977e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:19.599161 14258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c7ddd428afc4d3598f047b4befeb17b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:19.599143 14085 tablet_service.cc:1467] Processing CreateTablet for tablet 2c7ddd428afc4d3598f047b4befeb17b (DEFAULT_TABLE table=test-table [id=0936eae5a27d470ab920901b8d45977e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:19.599263 14085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c7ddd428afc4d3598f047b4befeb17b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:19.599408 14525 ts_tablet_manager.cc:1918] T 3485c69bbae542329c77e9277e5a0f41 P 2b9be7562599461aac701ceb1544a926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 14:58:19.599421 14522 ts_tablet_manager.cc:1918] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 14:58:19.599462 14525 log.cc:1196] T 3485c69bbae542329c77e9277e5a0f41 P 2b9be7562599461aac701ceb1544a926: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-0/wal/wals/3485c69bbae542329c77e9277e5a0f41
I20260806 14:58:19.599483 14522 log.cc:1196] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-1/wal/wals/3485c69bbae542329c77e9277e5a0f41
I20260806 14:58:19.599680 14525 ts_tablet_manager.cc:1939] T 3485c69bbae542329c77e9277e5a0f41 P 2b9be7562599461aac701ceb1544a926: Deleting consensus metadata
I20260806 14:58:19.599704 14522 ts_tablet_manager.cc:1939] T 3485c69bbae542329c77e9277e5a0f41 P c3bf26a2ee61406f9eca9b224c603adc: Deleting consensus metadata
I20260806 14:58:19.599843 13901 tablet_service.cc:1467] Processing CreateTablet for tablet 2c7ddd428afc4d3598f047b4befeb17b (DEFAULT_TABLE table=test-table [id=0936eae5a27d470ab920901b8d45977e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:19.599928 13901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c7ddd428afc4d3598f047b4befeb17b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:19.600522 14536 tablet_bootstrap.cc:492] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc: Bootstrap starting.
I20260806 14:58:19.600747 14538 tablet_bootstrap.cc:492] T 2c7ddd428afc4d3598f047b4befeb17b P 66922775c78f4c65926895f9744e03a4: Bootstrap starting.
I20260806 14:58:19.600837 14536 tablet_bootstrap.cc:654] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:19.601061 14538 tablet_bootstrap.cc:654] T 2c7ddd428afc4d3598f047b4befeb17b P 66922775c78f4c65926895f9744e03a4: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:19.601071 14537 tablet_bootstrap.cc:492] T 2c7ddd428afc4d3598f047b4befeb17b P 2b9be7562599461aac701ceb1544a926: Bootstrap starting.
I20260806 14:58:19.601486 14537 tablet_bootstrap.cc:654] T 2c7ddd428afc4d3598f047b4befeb17b P 2b9be7562599461aac701ceb1544a926: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:19.601519 13685 catalog_manager.cc:4841] TS c3bf26a2ee61406f9eca9b224c603adc (127.11.8.66:41741): tablet 3485c69bbae542329c77e9277e5a0f41 (table test-table [id=fe06831a1b904658b75717396957bdcf]) successfully deleted
I20260806 14:58:19.601822 14536 tablet_bootstrap.cc:492] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc: No bootstrap required, opened a new log
I20260806 14:58:19.601902 14536 ts_tablet_manager.cc:1397] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:19.601903 13686 catalog_manager.cc:4841] TS 2b9be7562599461aac701ceb1544a926 (127.11.8.65:36799): tablet 3485c69bbae542329c77e9277e5a0f41 (table test-table [id=fe06831a1b904658b75717396957bdcf]) successfully deleted
I20260806 14:58:19.602011 14536 raft_consensus.cc:348] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:19.602088 14536 raft_consensus.cc:374] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:19.602118 14536 raft_consensus.cc:729] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3bf26a2ee61406f9eca9b224c603adc, State: Initialized, Role: FOLLOWER
I20260806 14:58:19.602113 14537 tablet_bootstrap.cc:492] T 2c7ddd428afc4d3598f047b4befeb17b P 2b9be7562599461aac701ceb1544a926: No bootstrap required, opened a new log
I20260806 14:58:19.602138 14523 ts_tablet_manager.cc:1905] T 3485c69bbae542329c77e9277e5a0f41 P 66922775c78f4c65926895f9744e03a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:19.602159 14537 ts_tablet_manager.cc:1397] T 2c7ddd428afc4d3598f047b4befeb17b P 2b9be7562599461aac701ceb1544a926: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:19.602164 14536 consensus_queue.cc:260] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:19.602265 14536 ts_tablet_manager.cc:1428] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 14:58:19.602315 14537 raft_consensus.cc:348] T 2c7ddd428afc4d3598f047b4befeb17b P 2b9be7562599461aac701ceb1544a926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:19.602391 14537 raft_consensus.cc:374] T 2c7ddd428afc4d3598f047b4befeb17b P 2b9be7562599461aac701ceb1544a926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:19.602435 14537 raft_consensus.cc:729] T 2c7ddd428afc4d3598f047b4befeb17b P 2b9be7562599461aac701ceb1544a926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b9be7562599461aac701ceb1544a926, State: Initialized, Role: FOLLOWER
I20260806 14:58:19.602480 14537 consensus_queue.cc:260] T 2c7ddd428afc4d3598f047b4befeb17b P 2b9be7562599461aac701ceb1544a926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:19.602568 14537 ts_tablet_manager.cc:1428] T 2c7ddd428afc4d3598f047b4befeb17b P 2b9be7562599461aac701ceb1544a926: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 14:58:19.602726 14538 tablet_bootstrap.cc:492] T 2c7ddd428afc4d3598f047b4befeb17b P 66922775c78f4c65926895f9744e03a4: No bootstrap required, opened a new log
I20260806 14:58:19.602806 14538 ts_tablet_manager.cc:1397] T 2c7ddd428afc4d3598f047b4befeb17b P 66922775c78f4c65926895f9744e03a4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 14:58:19.602918 14538 raft_consensus.cc:348] T 2c7ddd428afc4d3598f047b4befeb17b P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:19.603016 14538 raft_consensus.cc:374] T 2c7ddd428afc4d3598f047b4befeb17b P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:19.603045 14538 raft_consensus.cc:729] T 2c7ddd428afc4d3598f047b4befeb17b P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66922775c78f4c65926895f9744e03a4, State: Initialized, Role: FOLLOWER
I20260806 14:58:19.603091 14538 consensus_queue.cc:260] T 2c7ddd428afc4d3598f047b4befeb17b P 66922775c78f4c65926895f9744e03a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:19.603190 14538 ts_tablet_manager.cc:1428] T 2c7ddd428afc4d3598f047b4befeb17b P 66922775c78f4c65926895f9744e03a4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 14:58:19.603610 14523 ts_tablet_manager.cc:1918] T 3485c69bbae542329c77e9277e5a0f41 P 66922775c78f4c65926895f9744e03a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 14:58:19.603668 14523 log.cc:1196] T 3485c69bbae542329c77e9277e5a0f41 P 66922775c78f4c65926895f9744e03a4: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-2/wal/wals/3485c69bbae542329c77e9277e5a0f41
I20260806 14:58:19.603933 14523 ts_tablet_manager.cc:1939] T 3485c69bbae542329c77e9277e5a0f41 P 66922775c78f4c65926895f9744e03a4: Deleting consensus metadata
I20260806 14:58:19.604257 13687 catalog_manager.cc:4841] TS 66922775c78f4c65926895f9744e03a4 (127.11.8.67:40001): tablet 3485c69bbae542329c77e9277e5a0f41 (table test-table [id=fe06831a1b904658b75717396957bdcf]) successfully deleted
I20260806 14:58:19.660774 14511 raft_consensus.cc:482] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 14:58:19.660884 14511 raft_consensus.cc:504] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:19.661093 14511 leader_election.cc:288] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b9be7562599461aac701ceb1544a926 (127.11.8.65:36799), 66922775c78f4c65926895f9744e03a4 (127.11.8.67:40001)
I20260806 14:58:19.661336 14306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c7ddd428afc4d3598f047b4befeb17b" candidate_uuid: "c3bf26a2ee61406f9eca9b224c603adc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66922775c78f4c65926895f9744e03a4" is_pre_election: true
I20260806 14:58:19.661336 13926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c7ddd428afc4d3598f047b4befeb17b" candidate_uuid: "c3bf26a2ee61406f9eca9b224c603adc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b9be7562599461aac701ceb1544a926" is_pre_election: true
I20260806 14:58:19.661454 13926 raft_consensus.cc:2454] T 2c7ddd428afc4d3598f047b4befeb17b P 2b9be7562599461aac701ceb1544a926 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3bf26a2ee61406f9eca9b224c603adc in term 0.
I20260806 14:58:19.661454 14306 raft_consensus.cc:2454] T 2c7ddd428afc4d3598f047b4befeb17b P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c3bf26a2ee61406f9eca9b224c603adc in term 0.
I20260806 14:58:19.661619 14026 leader_election.cc:302] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66922775c78f4c65926895f9744e03a4, c3bf26a2ee61406f9eca9b224c603adc; no voters:
I20260806 14:58:19.661741 14511 raft_consensus.cc:2781] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 14:58:19.661787 14511 raft_consensus.cc:482] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 14:58:19.661808 14511 raft_consensus.cc:3037] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:19.662504 14511 raft_consensus.cc:504] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:19.662655 14511 leader_election.cc:288] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc [CANDIDATE]: Term 1 election: Requested vote from peers 2b9be7562599461aac701ceb1544a926 (127.11.8.65:36799), 66922775c78f4c65926895f9744e03a4 (127.11.8.67:40001)
I20260806 14:58:19.662804 13926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c7ddd428afc4d3598f047b4befeb17b" candidate_uuid: "c3bf26a2ee61406f9eca9b224c603adc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b9be7562599461aac701ceb1544a926"
I20260806 14:58:19.662847 14306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c7ddd428afc4d3598f047b4befeb17b" candidate_uuid: "c3bf26a2ee61406f9eca9b224c603adc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66922775c78f4c65926895f9744e03a4"
I20260806 14:58:19.662882 13926 raft_consensus.cc:3037] T 2c7ddd428afc4d3598f047b4befeb17b P 2b9be7562599461aac701ceb1544a926 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:19.662940 14306 raft_consensus.cc:3037] T 2c7ddd428afc4d3598f047b4befeb17b P 66922775c78f4c65926895f9744e03a4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:19.663511 13926 raft_consensus.cc:2454] T 2c7ddd428afc4d3598f047b4befeb17b P 2b9be7562599461aac701ceb1544a926 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3bf26a2ee61406f9eca9b224c603adc in term 1.
I20260806 14:58:19.663630 14306 raft_consensus.cc:2454] T 2c7ddd428afc4d3598f047b4befeb17b P 66922775c78f4c65926895f9744e03a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c3bf26a2ee61406f9eca9b224c603adc in term 1.
I20260806 14:58:19.663698 14024 leader_election.cc:302] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b9be7562599461aac701ceb1544a926, c3bf26a2ee61406f9eca9b224c603adc; no voters:
I20260806 14:58:19.663815 14511 raft_consensus.cc:2781] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:19.663885 14511 raft_consensus.cc:686] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc [term 1 LEADER]: Becoming Leader. State: Replica: c3bf26a2ee61406f9eca9b224c603adc, State: Running, Role: LEADER
I20260806 14:58:19.663933 14511 consensus_queue.cc:237] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } }
I20260806 14:58:19.664567 13701 catalog_manager.cc:5495] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc reported cstate change: term changed from 0 to 1, leader changed from <none> to c3bf26a2ee61406f9eca9b224c603adc (127.11.8.66). New cstate: current_term: 1 leader_uuid: "c3bf26a2ee61406f9eca9b224c603adc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c3bf26a2ee61406f9eca9b224c603adc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 } health_report { overall_health: UNKNOWN } } }
I20260806 14:58:19.681082 14306 raft_consensus.cc:1261] T 2c7ddd428afc4d3598f047b4befeb17b P 66922775c78f4c65926895f9744e03a4 [term 1 FOLLOWER]: Refusing update from remote peer c3bf26a2ee61406f9eca9b224c603adc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:19.681356 13926 raft_consensus.cc:1261] T 2c7ddd428afc4d3598f047b4befeb17b P 2b9be7562599461aac701ceb1544a926 [term 1 FOLLOWER]: Refusing update from remote peer c3bf26a2ee61406f9eca9b224c603adc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:19.681370 14511 consensus_queue.cc:1035] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc [LEADER]: Connected to new peer: Peer: permanent_uuid: "66922775c78f4c65926895f9744e03a4" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:19.681661 14511 consensus_queue.cc:1035] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b9be7562599461aac701ceb1544a926" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 36799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 14:58:19.687301 14500 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 14:58:21.205199 13703 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:58:21.205312 13703 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:58:21.206187 13703 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 09dc9af5f18845a3aab211a08457af34: Sending DeleteTablet for 3 replicas of tablet 2c7ddd428afc4d3598f047b4befeb17b
I20260806 14:58:21.206502 14258 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c7ddd428afc4d3598f047b4befeb17b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:21 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:44214
I20260806 14:58:21.206497 14085 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c7ddd428afc4d3598f047b4befeb17b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:21 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:60438
I20260806 14:58:21.206488 13901 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c7ddd428afc4d3598f047b4befeb17b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:21 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:40680
I20260806 14:58:21.206753 14594 tablet_replica.cc:330] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc: stopping tablet replica
I20260806 14:58:21.206753 14592 tablet_replica.cc:330] T 2c7ddd428afc4d3598f047b4befeb17b P 66922775c78f4c65926895f9744e03a4: stopping tablet replica
I20260806 14:58:21.206872 14594 raft_consensus.cc:2229] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc [term 1 LEADER]: Raft consensus shutting down.
I20260806 14:58:21.206887 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 13812
I20260806 14:58:21.206812 14592 raft_consensus.cc:2229] T 2c7ddd428afc4d3598f047b4befeb17b P 66922775c78f4c65926895f9744e03a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:58:21.212531 14594 raft_consensus.cc:2258] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:21.232970 14594 ts_tablet_manager.cc:1905] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:21.206961 14592 raft_consensus.cc:2258] T 2c7ddd428afc4d3598f047b4befeb17b P 66922775c78f4c65926895f9744e03a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:21.234866 14592 ts_tablet_manager.cc:1905] T 2c7ddd428afc4d3598f047b4befeb17b P 66922775c78f4c65926895f9744e03a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:21.235294 14594 ts_tablet_manager.cc:1918] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 14:58:21.235369 14594 log.cc:1196] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-1/wal/wals/2c7ddd428afc4d3598f047b4befeb17b
I20260806 14:58:21.235648 14594 ts_tablet_manager.cc:1939] T 2c7ddd428afc4d3598f047b4befeb17b P c3bf26a2ee61406f9eca9b224c603adc: Deleting consensus metadata
I20260806 14:58:21.236092 13685 catalog_manager.cc:4841] TS c3bf26a2ee61406f9eca9b224c603adc (127.11.8.66:41741): tablet 2c7ddd428afc4d3598f047b4befeb17b (table test-table [id=0936eae5a27d470ab920901b8d45977e]) successfully deleted
I20260806 14:58:21.236569 14592 ts_tablet_manager.cc:1918] T 2c7ddd428afc4d3598f047b4befeb17b P 66922775c78f4c65926895f9744e03a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 14:58:21.236652 14592 log.cc:1196] T 2c7ddd428afc4d3598f047b4befeb17b P 66922775c78f4c65926895f9744e03a4: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786028287744255-11297-0/minicluster-data/ts-2/wal/wals/2c7ddd428afc4d3598f047b4befeb17b
I20260806 14:58:21.236862 14592 ts_tablet_manager.cc:1939] T 2c7ddd428afc4d3598f047b4befeb17b P 66922775c78f4c65926895f9744e03a4: Deleting consensus metadata
I20260806 14:58:21.237706 13687 catalog_manager.cc:4841] TS 66922775c78f4c65926895f9744e03a4 (127.11.8.67:40001): tablet 2c7ddd428afc4d3598f047b4befeb17b (table test-table [id=0936eae5a27d470ab920901b8d45977e]) successfully deleted
W20260806 14:58:21.238309 13686 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.65:36799 (error 108)
I20260806 14:58:21.238624 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 14004
I20260806 14:58:21.244196 14596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:21.238759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:36799 (local address 127.0.0.1:40696)
0806 14:58:21.244103 (+ 5344us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:21.244159 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.65:36799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5313,"thread_start_us":66,"threads_started":1}
W20260806 14:58:21.244377 13686 catalog_manager.cc:4584] TS 2b9be7562599461aac701ceb1544a926 (127.11.8.65:36799): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2c7ddd428afc4d3598f047b4befeb17b: Network error: Client connection negotiation failed: client connection to 127.11.8.65:36799: connect: Connection refused (error 111)
I20260806 14:58:21.251398 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 14191
I20260806 14:58:21.262461 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 13661
2026-08-06T14:58:21Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6101 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 14:58:21 dist-test-slave-68xk krb5kdc[14599](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 14:58:21 dist-test-slave-68xk krb5kdc[14599](info): set up 2 sockets
Aug 06 14:58:21 dist-test-slave-68xk krb5kdc[14599](info): commencing operation
krb5kdc: starting...
W20260806 14:58:22.468091 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 14:58:22 dist-test-slave-68xk krb5kdc[14599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T14:58:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T14:58:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 14:58:22.593791 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:40379
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:41461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:40379
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:22.682252 14617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:22.682549 14617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:22.682585 14617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:22.683881 14617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 14:58:22.683933 14617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:22.683948 14617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:22.683962 14617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 14:58:22.683974 14617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 14:58:22.685511 14617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:41461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:40379
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:40379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:22.685721 14617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:22.685901 14617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:22.691108 14627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:22.691877 14617 server_base.cc:1034] running on GCE node
W20260806 14:58:22.692019 14629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:22.692135 14626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:22.692487 14617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:22.692842 14617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:22.694000 14617 hybrid_clock.cc:648] HybridClock initialized: now 1786028302693973 us; error 45 us; skew 500 ppm
Aug 06 14:58:22 dist-test-slave-68xk krb5kdc[14599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028302, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:22.696679 14617 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 14:58:22.697170 14617 webserver.cc:460] Webserver started at http://127.11.8.126:34649/ using document root <none> and password file <none>
I20260806 14:58:22.697408 14617 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:22.697453 14617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:22.697598 14617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:22.698797 14617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "fc7a3b6dd73a44089f6c7a1c28f07a5f"
format_stamp: "Formatted at 2026-08-06 14:58:22 on dist-test-slave-68xk"
server_key: "b3dfca8e0b715d2da0c591b75d78813d"
server_key_iv: "ebe7a44eb5d9a7b268bdc30cf14faa4f"
server_key_version: "encryptionkey@0"
I20260806 14:58:22.699173 14617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "fc7a3b6dd73a44089f6c7a1c28f07a5f"
format_stamp: "Formatted at 2026-08-06 14:58:22 on dist-test-slave-68xk"
server_key: "b3dfca8e0b715d2da0c591b75d78813d"
server_key_iv: "ebe7a44eb5d9a7b268bdc30cf14faa4f"
server_key_version: "encryptionkey@0"
I20260806 14:58:22.700646 14617 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:22.701427 14637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:22.701624 14617 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:22.701735 14617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "fc7a3b6dd73a44089f6c7a1c28f07a5f"
format_stamp: "Formatted at 2026-08-06 14:58:22 on dist-test-slave-68xk"
server_key: "b3dfca8e0b715d2da0c591b75d78813d"
server_key_iv: "ebe7a44eb5d9a7b268bdc30cf14faa4f"
server_key_version: "encryptionkey@0"
I20260806 14:58:22.701809 14617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:22.712101 14617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:22.714022 14617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:22.714182 14617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:22.718047 14617 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:40379
I20260806 14:58:22.718080 14702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:40379 every 8 connection(s)
I20260806 14:58:22.718508 14617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 14:58:22.718879 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 14617
I20260806 14:58:22.719000 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 14:58:22.719225 11297 external_mini_cluster.cc:1428] Setting key 99f5e0a4215b77078aefbb9d7752ab17
I20260806 14:58:22.719604 14704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:22.722397 14704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc7a3b6dd73a44089f6c7a1c28f07a5f: Bootstrap starting.
I20260806 14:58:22.723086 14704 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc7a3b6dd73a44089f6c7a1c28f07a5f: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:22.723390 14704 log.cc:824] T 00000000000000000000000000000000 P fc7a3b6dd73a44089f6c7a1c28f07a5f: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:22.724086 14704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc7a3b6dd73a44089f6c7a1c28f07a5f: No bootstrap required, opened a new log
Aug 06 14:58:22 dist-test-slave-68xk krb5kdc[14599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:22.726351 14704 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc7a3b6dd73a44089f6c7a1c28f07a5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7a3b6dd73a44089f6c7a1c28f07a5f" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40379 } }
I20260806 14:58:22.726547 14704 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc7a3b6dd73a44089f6c7a1c28f07a5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:22.726574 14704 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc7a3b6dd73a44089f6c7a1c28f07a5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc7a3b6dd73a44089f6c7a1c28f07a5f, State: Initialized, Role: FOLLOWER
I20260806 14:58:22.726750 14704 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc7a3b6dd73a44089f6c7a1c28f07a5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7a3b6dd73a44089f6c7a1c28f07a5f" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40379 } }
I20260806 14:58:22.726848 14704 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc7a3b6dd73a44089f6c7a1c28f07a5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 14:58:22.726897 14704 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc7a3b6dd73a44089f6c7a1c28f07a5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 14:58:22.726950 14704 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc7a3b6dd73a44089f6c7a1c28f07a5f [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:22.727648 14704 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc7a3b6dd73a44089f6c7a1c28f07a5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7a3b6dd73a44089f6c7a1c28f07a5f" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40379 } }
I20260806 14:58:22.727768 14704 leader_election.cc:302] T 00000000000000000000000000000000 P fc7a3b6dd73a44089f6c7a1c28f07a5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc7a3b6dd73a44089f6c7a1c28f07a5f; no voters:
I20260806 14:58:22.727958 14704 leader_election.cc:288] T 00000000000000000000000000000000 P fc7a3b6dd73a44089f6c7a1c28f07a5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 14:58:22.728025 14711 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc7a3b6dd73a44089f6c7a1c28f07a5f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:22.728264 14704 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc7a3b6dd73a44089f6c7a1c28f07a5f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 14:58:22.728262 14711 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc7a3b6dd73a44089f6c7a1c28f07a5f [term 1 LEADER]: Becoming Leader. State: Replica: fc7a3b6dd73a44089f6c7a1c28f07a5f, State: Running, Role: LEADER
I20260806 14:58:22.728464 14711 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc7a3b6dd73a44089f6c7a1c28f07a5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7a3b6dd73a44089f6c7a1c28f07a5f" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40379 } }
I20260806 14:58:22.729707 14712 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc7a3b6dd73a44089f6c7a1c28f07a5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc7a3b6dd73a44089f6c7a1c28f07a5f. Latest consensus state: current_term: 1 leader_uuid: "fc7a3b6dd73a44089f6c7a1c28f07a5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7a3b6dd73a44089f6c7a1c28f07a5f" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40379 } } }
I20260806 14:58:22.729997 14712 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc7a3b6dd73a44089f6c7a1c28f07a5f [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:22.729946 14713 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc7a3b6dd73a44089f6c7a1c28f07a5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc7a3b6dd73a44089f6c7a1c28f07a5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7a3b6dd73a44089f6c7a1c28f07a5f" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40379 } } }
I20260806 14:58:22.730276 14713 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc7a3b6dd73a44089f6c7a1c28f07a5f [sys.catalog]: This master's current role is: LEADER
W20260806 14:58:22.730280 14726 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fc7a3b6dd73a44089f6c7a1c28f07a5f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 14:58:22.730404 14726 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 14:58:22.730959 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:22.719811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55342 (local address 127.11.8.126:40379)
0806 14:58:22.720023 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:22.720031 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:22.720057 (+ 26us) server_negotiation.cc:407] Connection header received
0806 14:58:22.720715 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:22.720723 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:22.720774 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:22.720928 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:22.721606 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:22.722629 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:22.723475 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:22.723895 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:22.724020 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:22.726817 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:22.726821 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:22.726821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:22.726834 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:22.729035 (+ 2201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:22.729505 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:22.729509 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:22.729510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:22.729646 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:22.729886 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:22.729887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:22.729887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:22.730186 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:22.730330 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:22.730608 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 14:58:22.730713 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":100,"threads_started":1}
I20260806 14:58:22.734591 14729 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 14:58:22.735199 14729 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 14:58:22.736778 14729 catalog_manager.cc:1347] Generated new cluster ID: 0189a3de89914f87be30366a4e47f560
I20260806 14:58:22.736855 14729 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 14:58:22.774044 14729 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 14:58:22.774648 14729 catalog_manager.cc:1504] Loading token signing keys...
I20260806 14:58:22.784354 14729 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc7a3b6dd73a44089f6c7a1c28f07a5f: Generated new TSK 0
I20260806 14:58:22.784559 14729 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
I20260806 14:58:22.817914 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:0
--local_ip_for_outbound_sockets=127.11.8.65
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:40379
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:41461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:22.902231 14736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:22.902454 14736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:22.902478 14736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:22.903676 14736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:22.903718 14736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:22.903745 14736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 14:58:22.905121 14736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:41461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:40379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:22.905303 14736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:22.905474 14736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:22.910624 14744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:22.910632 14747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:22.910849 14736 server_base.cc:1034] running on GCE node
W20260806 14:58:22.910634 14745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:22.911154 14736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:22.911397 14736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:22.912562 14736 hybrid_clock.cc:648] HybridClock initialized: now 1786028302912547 us; error 35 us; skew 500 ppm
Aug 06 14:58:22 dist-test-slave-68xk krb5kdc[14599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028302, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:22.915160 14736 init.cc:377] Logged in from keytab as kudu/127.11.8.65@KRBTEST.COM (short username kudu)
I20260806 14:58:22.915584 14736 webserver.cc:460] Webserver started at http://127.11.8.65:40021/ using document root <none> and password file <none>
I20260806 14:58:22.915817 14736 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:22.915887 14736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:22.916070 14736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:22.917140 14736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/data/instance:
uuid: "101d770a458840ac99fc8b9df9ebdf47"
format_stamp: "Formatted at 2026-08-06 14:58:22 on dist-test-slave-68xk"
server_key: "ecf17a8f17e05c0be6e9b3f637d7da5f"
server_key_iv: "e9f80d56cde28cef462652e2bc468c83"
server_key_version: "encryptionkey@0"
I20260806 14:58:22.917519 14736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance:
uuid: "101d770a458840ac99fc8b9df9ebdf47"
format_stamp: "Formatted at 2026-08-06 14:58:22 on dist-test-slave-68xk"
server_key: "ecf17a8f17e05c0be6e9b3f637d7da5f"
server_key_iv: "e9f80d56cde28cef462652e2bc468c83"
server_key_version: "encryptionkey@0"
I20260806 14:58:22.918964 14736 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 14:58:22.919791 14755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:22.919991 14736 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:22.920099 14736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "101d770a458840ac99fc8b9df9ebdf47"
format_stamp: "Formatted at 2026-08-06 14:58:22 on dist-test-slave-68xk"
server_key: "ecf17a8f17e05c0be6e9b3f637d7da5f"
server_key_iv: "e9f80d56cde28cef462652e2bc468c83"
server_key_version: "encryptionkey@0"
I20260806 14:58:22.920176 14736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:22.931201 14736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:22.933147 14736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:22.933317 14736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:22.933557 14736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:22.933885 14736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:22.933941 14736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:22.933975 14736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:22.934005 14736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:22.940373 14736 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:40881
I20260806 14:58:22.940474 14913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:40881 every 8 connection(s)
I20260806 14:58:22.940794 14736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
I20260806 14:58:22.943511 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 14736
I20260806 14:58:22.943626 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance
I20260806 14:58:22.943828 11297 external_mini_cluster.cc:1428] Setting key c6db50a53dca7621ccc399dc1dfdf075
Aug 06 14:58:22 dist-test-slave-68xk krb5kdc[14599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028302, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:22.951406 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:22.941581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:55237 (local address 127.11.8.126:40379)
0806 14:58:22.941685 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:22.941687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:22.941875 (+ 188us) server_negotiation.cc:407] Connection header received
0806 14:58:22.942471 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:22.942472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:22.942485 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:22.942543 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:22.943501 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:22.944240 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:22.945336 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:22.945559 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:22.945648 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:22.948068 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:22.948070 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:22.948070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:22.948079 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:22.949846 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:22.950311 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:22.950312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:22.950313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:22.950381 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:22.950716 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:22.950717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:22.950718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:22.950868 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:22.950946 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:22.951213 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 14:58:22.951304 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 14:58:22.951543 14916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:22.941535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40379 (local address 127.11.8.65:55237)
0806 14:58:22.941733 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:22.941782 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:22.942334 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:22.942610 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:22.942614 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:22.943207 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:22.943396 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:22.943400 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:22.944334 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:22.944336 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:22.945241 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:22.945244 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:22.945659 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:22.945660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:22.945802 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:22.946385 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:22.946395 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:22.947971 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:22.949938 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:22.949945 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:22.949947 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:22.950191 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:22.950527 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:22.950528 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:22.950528 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:22.950641 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:22.950944 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:22.950948 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:22.951072 (+ 124us) client_negotiation.cc:769] Sending connection context
0806 14:58:22.951162 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 14:58:22.951325 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":99,"threads_started":1}
I20260806 14:58:22.952127 14914 heartbeater.cc:347] Connected to a master server at 127.11.8.126:40379
I20260806 14:58:22.952231 14914 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:22.952445 14914 heartbeater.cc:511] Master 127.11.8.126:40379 requested a full tablet report, sending...
I20260806 14:58:22.953039 14658 ts_manager.cc:194] Registered new tserver with Master: 101d770a458840ac99fc8b9df9ebdf47 (127.11.8.65:40881)
WARNING: no policy specified for kudu/127.11.8.66@KRBTEST.COM; defaulting to no policy
I20260806 14:58:22.953884 14658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.65@KRBTEST.COM'} at 127.11.8.65:55237
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
I20260806 14:58:22.973105 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:0
--local_ip_for_outbound_sockets=127.11.8.66
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:40379
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:41461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:23.060420 14920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:23.060655 14920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:23.060696 14920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:23.061867 14920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:23.061933 14920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:23.061980 14920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 14:58:23.063480 14920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:41461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:40379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:23.063762 14920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:23.063999 14920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:23.069183 14926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:23.069190 14925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:23.069271 14931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:23.069478 14920 server_base.cc:1034] running on GCE node
I20260806 14:58:23.069715 14920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:23.069974 14920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:23.071164 14920 hybrid_clock.cc:648] HybridClock initialized: now 1786028303071153 us; error 44 us; skew 500 ppm
Aug 06 14:58:23 dist-test-slave-68xk krb5kdc[14599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028303, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:23.073837 14920 init.cc:377] Logged in from keytab as kudu/127.11.8.66@KRBTEST.COM (short username kudu)
I20260806 14:58:23.074370 14920 webserver.cc:460] Webserver started at http://127.11.8.66:42623/ using document root <none> and password file <none>
I20260806 14:58:23.074647 14920 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:23.074726 14920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:23.074898 14920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:23.075991 14920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/data/instance:
uuid: "bd02a537f5de4132ab1fe58f846baa35"
format_stamp: "Formatted at 2026-08-06 14:58:23 on dist-test-slave-68xk"
server_key: "c71725e4f94f087b2bd9b0112c1738a6"
server_key_iv: "79dccd9d61e59641eeeedde6c915143f"
server_key_version: "encryptionkey@0"
I20260806 14:58:23.076494 14920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance:
uuid: "bd02a537f5de4132ab1fe58f846baa35"
format_stamp: "Formatted at 2026-08-06 14:58:23 on dist-test-slave-68xk"
server_key: "c71725e4f94f087b2bd9b0112c1738a6"
server_key_iv: "79dccd9d61e59641eeeedde6c915143f"
server_key_version: "encryptionkey@0"
I20260806 14:58:23.077927 14920 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 14:58:23.078792 14941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:23.079013 14920 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 14:58:23.079149 14920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "bd02a537f5de4132ab1fe58f846baa35"
format_stamp: "Formatted at 2026-08-06 14:58:23 on dist-test-slave-68xk"
server_key: "c71725e4f94f087b2bd9b0112c1738a6"
server_key_iv: "79dccd9d61e59641eeeedde6c915143f"
server_key_version: "encryptionkey@0"
I20260806 14:58:23.079231 14920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:23.085717 14920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:23.087708 14920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:23.087872 14920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:23.088100 14920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:23.088438 14920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:23.088495 14920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:23.088531 14920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:23.088557 14920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:23.094869 14920 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:41021
I20260806 14:58:23.094940 15101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:41021 every 8 connection(s)
I20260806 14:58:23.095256 14920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
I20260806 14:58:23.098567 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 14920
I20260806 14:58:23.098677 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance
I20260806 14:58:23.098876 11297 external_mini_cluster.cc:1428] Setting key ed3d0fced365225101f39a3b063d128c
Aug 06 14:58:23 dist-test-slave-68xk krb5kdc[14599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028303, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:23.105410 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:23.096030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:48897 (local address 127.11.8.126:40379)
0806 14:58:23.096103 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:23.096105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:23.096287 (+ 182us) server_negotiation.cc:407] Connection header received
0806 14:58:23.096778 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:23.096779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:23.096791 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:23.096844 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:23.097816 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:23.098327 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:23.099642 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:23.099946 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:23.100022 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:23.102327 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:23.102328 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:23.102329 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:23.102343 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:23.103939 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:23.104379 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:23.104380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:23.104381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:23.104446 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:23.104735 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:23.104736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:23.104737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:23.104857 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:23.104930 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:23.105237 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 14:58:23.105320 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 14:58:23.105609 15107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:23.096011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40379 (local address 127.11.8.66:48897)
0806 14:58:23.096174 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:23.096210 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:23.096689 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:23.096897 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:23.096901 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:23.097503 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:23.097732 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:23.097736 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:23.098485 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:23.098486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:23.099569 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:23.099574 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:23.100104 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:23.100106 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:23.100214 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:23.100780 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:23.100790 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:23.102210 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:23.104051 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:23.104059 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:23.104061 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:23.104295 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:23.104518 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:23.104519 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:23.104520 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:23.104651 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:23.104933 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:23.104941 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:23.105094 (+ 153us) client_negotiation.cc:769] Sending connection context
0806 14:58:23.105203 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 14:58:23.105365 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
I20260806 14:58:23.106134 15103 heartbeater.cc:347] Connected to a master server at 127.11.8.126:40379
I20260806 14:58:23.106262 15103 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:23.106492 15103 heartbeater.cc:511] Master 127.11.8.126:40379 requested a full tablet report, sending...
I20260806 14:58:23.106930 14658 ts_manager.cc:194] Registered new tserver with Master: bd02a537f5de4132ab1fe58f846baa35 (127.11.8.66:41021)
I20260806 14:58:23.107460 14658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.66@KRBTEST.COM'} at 127.11.8.66:48897
WARNING: no policy specified for kudu/127.11.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
I20260806 14:58:23.125468 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:0
--local_ip_for_outbound_sockets=127.11.8.67
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:40379
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:41461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:23.208956 15110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:23.209167 15110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:23.209192 15110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:23.210304 15110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:23.210342 15110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:23.210368 15110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 14:58:23.211934 15110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:41461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:40379
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:23.212184 15110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:23.212419 15110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:23.217628 15119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:23.217636 15117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:23.217636 15116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:23.217669 15110 server_base.cc:1034] running on GCE node
I20260806 14:58:23.218055 15110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:23.218315 15110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:23.219566 15110 hybrid_clock.cc:648] HybridClock initialized: now 1786028303219551 us; error 49 us; skew 500 ppm
Aug 06 14:58:23 dist-test-slave-68xk krb5kdc[14599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028303, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:23.222057 15110 init.cc:377] Logged in from keytab as kudu/127.11.8.67@KRBTEST.COM (short username kudu)
I20260806 14:58:23.222584 15110 webserver.cc:460] Webserver started at http://127.11.8.67:42815/ using document root <none> and password file <none>
I20260806 14:58:23.222776 15110 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:23.222818 15110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:23.222909 15110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:23.223973 15110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/data/instance:
uuid: "8a57ba8a4e1f45b2a938bb698036814a"
format_stamp: "Formatted at 2026-08-06 14:58:23 on dist-test-slave-68xk"
server_key: "a009d8bdd8583cdd8b4f8f1fb71fdd65"
server_key_iv: "bd7408da73e6073a5fa7f3039ea74120"
server_key_version: "encryptionkey@0"
I20260806 14:58:23.224289 15110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance:
uuid: "8a57ba8a4e1f45b2a938bb698036814a"
format_stamp: "Formatted at 2026-08-06 14:58:23 on dist-test-slave-68xk"
server_key: "a009d8bdd8583cdd8b4f8f1fb71fdd65"
server_key_iv: "bd7408da73e6073a5fa7f3039ea74120"
server_key_version: "encryptionkey@0"
I20260806 14:58:23.225674 15110 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 14:58:23.226503 15129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:23.226686 15110 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:23.226796 15110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "8a57ba8a4e1f45b2a938bb698036814a"
format_stamp: "Formatted at 2026-08-06 14:58:23 on dist-test-slave-68xk"
server_key: "a009d8bdd8583cdd8b4f8f1fb71fdd65"
server_key_iv: "bd7408da73e6073a5fa7f3039ea74120"
server_key_version: "encryptionkey@0"
I20260806 14:58:23.226881 15110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:23.235316 15110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:23.237298 15110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:23.237461 15110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:23.237696 15110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:23.238044 15110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:23.238104 15110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:23.238147 15110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:23.238174 15110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:23.244339 15110 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:45021
I20260806 14:58:23.244392 15277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:45021 every 8 connection(s)
I20260806 14:58:23.244751 15110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
Aug 06 14:58:23 dist-test-slave-68xk krb5kdc[14599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028303, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:23.251191 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 15110
I20260806 14:58:23.251302 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance
I20260806 14:58:23.251474 11297 external_mini_cluster.cc:1428] Setting key 8a23f297f27216f7a165a5359d35f74f
I20260806 14:58:23.255098 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:23.245531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:57391 (local address 127.11.8.126:40379)
0806 14:58:23.245601 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:23.245603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:23.245830 (+ 227us) server_negotiation.cc:407] Connection header received
0806 14:58:23.246341 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:23.246342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:23.246352 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:23.246452 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:23.247513 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:23.248020 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:23.249090 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:23.249299 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:23.249356 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:23.251899 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:23.251901 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:23.251901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:23.251909 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:23.253600 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:23.253965 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:23.253966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:23.253966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:23.254022 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:23.254466 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:23.254468 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:23.254468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:23.254601 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:23.254663 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:23.254923 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 14:58:23.254991 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 14:58:23.255218 15280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:23.245512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40379 (local address 127.11.8.67:57391)
0806 14:58:23.245700 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:23.245737 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:23.246253 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:23.246525 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:23.246528 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:23.247154 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:23.247354 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:23.247359 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:23.248132 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:23.248133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:23.249002 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:23.249004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:23.249379 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:23.249380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:23.249486 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:23.250099 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:23.250109 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:23.251799 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:23.253674 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:23.253679 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:23.253681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:23.253899 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:23.254160 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:23.254162 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:23.254162 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:23.254351 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:23.254672 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:23.254678 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:23.254796 (+ 118us) client_negotiation.cc:769] Sending connection context
0806 14:58:23.254894 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 14:58:23.255022 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":112,"threads_started":1}
I20260806 14:58:23.255724 15278 heartbeater.cc:347] Connected to a master server at 127.11.8.126:40379
I20260806 14:58:23.255836 15278 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:23.256027 15278 heartbeater.cc:511] Master 127.11.8.126:40379 requested a full tablet report, sending...
I20260806 14:58:23.256434 14658 ts_manager.cc:194] Registered new tserver with Master: 8a57ba8a4e1f45b2a938bb698036814a (127.11.8.67:45021)
I20260806 14:58:23.256780 14658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.67@KRBTEST.COM'} at 127.11.8.67:57391
I20260806 14:58:23.263337 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 14:58:23.276654 14708 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:23.275576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55356 (local address 127.11.8.126:40379)
0806 14:58:23.275674 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:23.275676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:23.275819 (+ 143us) server_negotiation.cc:407] Connection header received
0806 14:58:23.275855 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:23.275856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:23.275866 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:23.275919 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:23.276590 (+ 671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55356: BlockingRecv error: recv got EOF from 127.0.0.1:55356 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 14:58:23.277668 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 14736
I20260806 14:58:23.296386 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 14920
I20260806 14:58:23.307775 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 15110
I20260806 14:58:23.315670 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 14617
2026-08-06T14:58:23Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2047 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 14:58:23 dist-test-slave-68xk krb5kdc[15295](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 14:58:23 dist-test-slave-68xk krb5kdc[15295](info): set up 2 sockets
Aug 06 14:58:23 dist-test-slave-68xk krb5kdc[15295](info): commencing operation
krb5kdc: starting...
W20260806 14:58:24.534760 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 14:58:24 dist-test-slave-68xk krb5kdc[15295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T14:58:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T14:58:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 14:58:24.663375 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:39683
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:39609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:39683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:24.748576 15315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:24.748858 15315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:24.748905 15315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:24.750139 15315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 14:58:24.750214 15315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:24.750239 15315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:24.750262 15315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 14:58:24.750290 15315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 14:58:24.751820 15315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:39609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:39683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:39683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:24.752125 15315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:24.752363 15315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:24.757403 15323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:24.757385 15324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:24.757416 15315 server_base.cc:1034] running on GCE node
W20260806 14:58:24.757380 15327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:24.757911 15315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:24.758257 15315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:24.759433 15315 hybrid_clock.cc:648] HybridClock initialized: now 1786028304759416 us; error 55 us; skew 500 ppm
Aug 06 14:58:24 dist-test-slave-68xk krb5kdc[15295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028304, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:24.767563 15315 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 14:58:24.768088 15315 webserver.cc:460] Webserver started at http://127.11.8.126:37541/ using document root <none> and password file <none>
I20260806 14:58:24.768348 15315 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:24.768397 15315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:24.768575 15315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:24.769677 15315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "2722e44e2f1649d0a24e08d3a0aa4118"
format_stamp: "Formatted at 2026-08-06 14:58:24 on dist-test-slave-68xk"
server_key: "a150be361729602d5895310cc449d234"
server_key_iv: "68a835ef5fd99697f7ca756536185ca2"
server_key_version: "encryptionkey@0"
I20260806 14:58:24.770056 15315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "2722e44e2f1649d0a24e08d3a0aa4118"
format_stamp: "Formatted at 2026-08-06 14:58:24 on dist-test-slave-68xk"
server_key: "a150be361729602d5895310cc449d234"
server_key_iv: "68a835ef5fd99697f7ca756536185ca2"
server_key_version: "encryptionkey@0"
I20260806 14:58:24.771509 15315 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 14:58:24.772265 15337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:24.772464 15315 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 14:58:24.772563 15315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "2722e44e2f1649d0a24e08d3a0aa4118"
format_stamp: "Formatted at 2026-08-06 14:58:24 on dist-test-slave-68xk"
server_key: "a150be361729602d5895310cc449d234"
server_key_iv: "68a835ef5fd99697f7ca756536185ca2"
server_key_version: "encryptionkey@0"
I20260806 14:58:24.772612 15315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:24.788905 15315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:24.790812 15315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:24.790921 15315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:24.794740 15315 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:39683
I20260806 14:58:24.794790 15403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:39683 every 8 connection(s)
I20260806 14:58:24.795361 15315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 14:58:24.796068 15404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:24.798159 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 15315
I20260806 14:58:24.798233 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 14:58:24.798448 11297 external_mini_cluster.cc:1428] Setting key 8b7a941c3d034a0772bf1b26ee63f81e
I20260806 14:58:24.798507 15404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2722e44e2f1649d0a24e08d3a0aa4118: Bootstrap starting.
I20260806 14:58:24.799227 15404 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2722e44e2f1649d0a24e08d3a0aa4118: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:24.799551 15404 log.cc:824] T 00000000000000000000000000000000 P 2722e44e2f1649d0a24e08d3a0aa4118: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:24.800366 15404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2722e44e2f1649d0a24e08d3a0aa4118: No bootstrap required, opened a new log
I20260806 14:58:24.802642 15404 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2722e44e2f1649d0a24e08d3a0aa4118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2722e44e2f1649d0a24e08d3a0aa4118" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 39683 } }
I20260806 14:58:24.802831 15404 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2722e44e2f1649d0a24e08d3a0aa4118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:24.802886 15404 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2722e44e2f1649d0a24e08d3a0aa4118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2722e44e2f1649d0a24e08d3a0aa4118, State: Initialized, Role: FOLLOWER
I20260806 14:58:24.803020 15404 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2722e44e2f1649d0a24e08d3a0aa4118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2722e44e2f1649d0a24e08d3a0aa4118" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 39683 } }
I20260806 14:58:24.803119 15404 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2722e44e2f1649d0a24e08d3a0aa4118 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 14:58:24.803164 15404 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2722e44e2f1649d0a24e08d3a0aa4118 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 14:58:24.803213 15404 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2722e44e2f1649d0a24e08d3a0aa4118 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:24.803926 15404 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2722e44e2f1649d0a24e08d3a0aa4118 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2722e44e2f1649d0a24e08d3a0aa4118" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 39683 } }
I20260806 14:58:24.804046 15404 leader_election.cc:302] T 00000000000000000000000000000000 P 2722e44e2f1649d0a24e08d3a0aa4118 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2722e44e2f1649d0a24e08d3a0aa4118; no voters:
I20260806 14:58:24.804214 15404 leader_election.cc:288] T 00000000000000000000000000000000 P 2722e44e2f1649d0a24e08d3a0aa4118 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 14:58:24.804270 15414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2722e44e2f1649d0a24e08d3a0aa4118 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:24.804414 15414 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2722e44e2f1649d0a24e08d3a0aa4118 [term 1 LEADER]: Becoming Leader. State: Replica: 2722e44e2f1649d0a24e08d3a0aa4118, State: Running, Role: LEADER
I20260806 14:58:24.804522 15404 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2722e44e2f1649d0a24e08d3a0aa4118 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 14:58:24.804534 15414 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2722e44e2f1649d0a24e08d3a0aa4118 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2722e44e2f1649d0a24e08d3a0aa4118" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 39683 } }
I20260806 14:58:24.804869 15414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2722e44e2f1649d0a24e08d3a0aa4118 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2722e44e2f1649d0a24e08d3a0aa4118. Latest consensus state: current_term: 1 leader_uuid: "2722e44e2f1649d0a24e08d3a0aa4118" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2722e44e2f1649d0a24e08d3a0aa4118" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 39683 } } }
I20260806 14:58:24.804947 15414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2722e44e2f1649d0a24e08d3a0aa4118 [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:24.805125 15416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2722e44e2f1649d0a24e08d3a0aa4118 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2722e44e2f1649d0a24e08d3a0aa4118" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2722e44e2f1649d0a24e08d3a0aa4118" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 39683 } } }
I20260806 14:58:24.805200 15416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2722e44e2f1649d0a24e08d3a0aa4118 [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:24.805327 15421 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 06 14:58:24 dist-test-slave-68xk krb5kdc[15295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:24.806025 15421 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 14:58:24.809593 15412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:24.798962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46808 (local address 127.11.8.126:39683)
0806 14:58:24.799129 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:24.799139 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:24.799172 (+ 33us) server_negotiation.cc:407] Connection header received
0806 14:58:24.799691 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:24.799703 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:24.799781 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:24.799930 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:24.800788 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:24.801619 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:24.802794 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:24.803384 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:24.803466 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:24.806498 (+ 3032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:24.806503 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:24.806503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:24.806517 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:24.808095 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:24.808459 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:24.808463 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:24.808463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:24.808561 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:24.808772 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:24.808774 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:24.808774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:24.809012 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:24.809159 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:24.809279 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 14:58:24.809413 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
I20260806 14:58:24.810640 11297 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 14:58:24.811478 15421 catalog_manager.cc:1347] Generated new cluster ID: 356cf205d7ca4ccc93742021ce10a8f7
I20260806 14:58:24.811532 15421 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 14:58:24.826193 15412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:24.822802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46822 (local address 127.11.8.126:39683)
0806 14:58:24.822845 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:24.822847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:24.822973 (+ 126us) server_negotiation.cc:407] Connection header received
0806 14:58:24.823051 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:24.823052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:24.823064 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:24.823129 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:24.823487 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:24.823963 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:24.824676 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:24.824906 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:24.824985 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:24.825687 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:24.825688 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:24.825689 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 14:58:24.825898 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:24.825914 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 14:58:24.825921 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:24.825993 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:24.826059 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 14:58:24.826083 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 14:58:24.845566 15421 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 14:58:24.846395 15421 catalog_manager.cc:1504] Loading token signing keys...
I20260806 14:58:24.856164 15421 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2722e44e2f1649d0a24e08d3a0aa4118: Generated new TSK 0
I20260806 14:58:24.856392 15421 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 14:58:24.864355 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 15315
2026-08-06T14:58:24Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1549 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 14:58:24 dist-test-slave-68xk krb5kdc[15450](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 14:58:24 dist-test-slave-68xk krb5kdc[15450](info): set up 2 sockets
Aug 06 14:58:24 dist-test-slave-68xk krb5kdc[15450](info): commencing operation
krb5kdc: starting...
W20260806 14:58:26.129381 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 14:58:26 dist-test-slave-68xk krb5kdc[15450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T14:58:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T14:58:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 14:58:26.256906 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:40517
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:42199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:40517
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:26.343777 15469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:26.343973 15469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:26.343995 15469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:26.345055 15469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 14:58:26.345089 15469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:26.345103 15469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:26.345115 15469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 14:58:26.345129 15469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 14:58:26.346530 15469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:42199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:40517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:40517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:26.346720 15469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:26.346881 15469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:26.351724 15478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:26.351751 15476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:26.351770 15469 server_base.cc:1034] running on GCE node
W20260806 14:58:26.351751 15481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:26.352207 15469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:26.352535 15469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:26.353710 15469 hybrid_clock.cc:648] HybridClock initialized: now 1786028306353696 us; error 44 us; skew 500 ppm
Aug 06 14:58:26 dist-test-slave-68xk krb5kdc[15450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028306, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:26.356249 15469 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 14:58:26.356777 15469 webserver.cc:460] Webserver started at http://127.11.8.126:42127/ using document root <none> and password file <none>
I20260806 14:58:26.357007 15469 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:26.357077 15469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:26.357249 15469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:26.358292 15469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "778a9511531c462e81041fb38832a133"
format_stamp: "Formatted at 2026-08-06 14:58:26 on dist-test-slave-68xk"
server_key: "4561028078ad72c337dd95803effba09"
server_key_iv: "46edb01f1b15775f5ec1531b6298cc6d"
server_key_version: "encryptionkey@0"
I20260806 14:58:26.358708 15469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "778a9511531c462e81041fb38832a133"
format_stamp: "Formatted at 2026-08-06 14:58:26 on dist-test-slave-68xk"
server_key: "4561028078ad72c337dd95803effba09"
server_key_iv: "46edb01f1b15775f5ec1531b6298cc6d"
server_key_version: "encryptionkey@0"
I20260806 14:58:26.360055 15469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:26.360790 15492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:26.360975 15469 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 14:58:26.361086 15469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "778a9511531c462e81041fb38832a133"
format_stamp: "Formatted at 2026-08-06 14:58:26 on dist-test-slave-68xk"
server_key: "4561028078ad72c337dd95803effba09"
server_key_iv: "46edb01f1b15775f5ec1531b6298cc6d"
server_key_version: "encryptionkey@0"
I20260806 14:58:26.361161 15469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:26.384824 15469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:26.386766 15469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:26.386934 15469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:26.391021 15469 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:40517
I20260806 14:58:26.391057 15560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:40517 every 8 connection(s)
I20260806 14:58:26.391415 15469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 14:58:26.392282 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 15469
I20260806 14:58:26.392393 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 14:58:26.392591 11297 external_mini_cluster.cc:1428] Setting key 6f4b28aa528758e91df7bfaa14d59023
I20260806 14:58:26.392745 15562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:26.396204 15562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 778a9511531c462e81041fb38832a133: Bootstrap starting.
I20260806 14:58:26.396932 15562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 778a9511531c462e81041fb38832a133: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:26.397254 15562 log.cc:824] T 00000000000000000000000000000000 P 778a9511531c462e81041fb38832a133: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:26.398062 15562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 778a9511531c462e81041fb38832a133: No bootstrap required, opened a new log
Aug 06 14:58:26 dist-test-slave-68xk krb5kdc[15450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:26.400431 15562 raft_consensus.cc:348] T 00000000000000000000000000000000 P 778a9511531c462e81041fb38832a133 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "778a9511531c462e81041fb38832a133" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40517 } }
I20260806 14:58:26.400611 15562 raft_consensus.cc:374] T 00000000000000000000000000000000 P 778a9511531c462e81041fb38832a133 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:26.400653 15562 raft_consensus.cc:729] T 00000000000000000000000000000000 P 778a9511531c462e81041fb38832a133 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 778a9511531c462e81041fb38832a133, State: Initialized, Role: FOLLOWER
I20260806 14:58:26.400799 15562 consensus_queue.cc:260] T 00000000000000000000000000000000 P 778a9511531c462e81041fb38832a133 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "778a9511531c462e81041fb38832a133" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40517 } }
I20260806 14:58:26.400892 15562 raft_consensus.cc:388] T 00000000000000000000000000000000 P 778a9511531c462e81041fb38832a133 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 14:58:26.400951 15562 raft_consensus.cc:482] T 00000000000000000000000000000000 P 778a9511531c462e81041fb38832a133 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 14:58:26.401006 15562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 778a9511531c462e81041fb38832a133 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:26.401710 15562 raft_consensus.cc:504] T 00000000000000000000000000000000 P 778a9511531c462e81041fb38832a133 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "778a9511531c462e81041fb38832a133" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40517 } }
I20260806 14:58:26.401820 15562 leader_election.cc:302] T 00000000000000000000000000000000 P 778a9511531c462e81041fb38832a133 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 778a9511531c462e81041fb38832a133; no voters:
I20260806 14:58:26.401980 15562 leader_election.cc:288] T 00000000000000000000000000000000 P 778a9511531c462e81041fb38832a133 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 14:58:26.402040 15573 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 778a9511531c462e81041fb38832a133 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:26.402202 15573 raft_consensus.cc:686] T 00000000000000000000000000000000 P 778a9511531c462e81041fb38832a133 [term 1 LEADER]: Becoming Leader. State: Replica: 778a9511531c462e81041fb38832a133, State: Running, Role: LEADER
I20260806 14:58:26.402254 15562 sys_catalog.cc:564] T 00000000000000000000000000000000 P 778a9511531c462e81041fb38832a133 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 14:58:26.402300 15573 consensus_queue.cc:237] T 00000000000000000000000000000000 P 778a9511531c462e81041fb38832a133 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "778a9511531c462e81041fb38832a133" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40517 } }
I20260806 14:58:26.402638 15573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 778a9511531c462e81041fb38832a133 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 778a9511531c462e81041fb38832a133. Latest consensus state: current_term: 1 leader_uuid: "778a9511531c462e81041fb38832a133" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "778a9511531c462e81041fb38832a133" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40517 } } }
I20260806 14:58:26.402729 15573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 778a9511531c462e81041fb38832a133 [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:26.402645 15574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 778a9511531c462e81041fb38832a133 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "778a9511531c462e81041fb38832a133" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "778a9511531c462e81041fb38832a133" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40517 } } }
I20260806 14:58:26.402881 15574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 778a9511531c462e81041fb38832a133 [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:26.402953 15581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 14:58:26.403489 15581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 14:58:26.404477 15565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:26.393175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55528 (local address 127.11.8.126:40517)
0806 14:58:26.393385 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:26.393401 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:26.393426 (+ 25us) server_negotiation.cc:407] Connection header received
0806 14:58:26.393899 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:26.393906 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:26.393951 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:26.394062 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:26.394913 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:26.395813 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:26.396871 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:26.397338 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:26.397466 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:26.400049 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:26.400054 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:26.400055 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:26.400070 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:26.402183 (+ 2113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:26.402668 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:26.402671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:26.402672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:26.402816 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:26.403090 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:26.403090 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:26.403091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:26.403406 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:26.404183 (+ 777us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:26.404240 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 14:58:26.404339 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":105,"threads_started":1}
I20260806 14:58:26.405699 15581 catalog_manager.cc:1347] Generated new cluster ID: 87532be3c2a04c44af7e4320eed0aadb
I20260806 14:58:26.405772 15581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 14:58:26.420892 15581 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 14:58:26.421459 15581 catalog_manager.cc:1504] Loading token signing keys...
I20260806 14:58:26.428452 15581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 778a9511531c462e81041fb38832a133: Generated new TSK 0
I20260806 14:58:26.428668 15581 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
I20260806 14:58:26.464944 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:0
--local_ip_for_outbound_sockets=127.11.8.65
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:40517
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:42199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:26.563997 15601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:26.564201 15601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:26.564232 15601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:26.565476 15601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:26.565518 15601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:26.565544 15601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 14:58:26.567035 15601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:42199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:40517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:26.567306 15601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:26.567541 15601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:26.572886 15612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:26.572896 15614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:26.573153 15601 server_base.cc:1034] running on GCE node
W20260806 14:58:26.573220 15610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:26.573553 15601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:26.573941 15601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:26.575130 15601 hybrid_clock.cc:648] HybridClock initialized: now 1786028306575107 us; error 42 us; skew 500 ppm
Aug 06 14:58:26 dist-test-slave-68xk krb5kdc[15450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028306, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:26.577687 15601 init.cc:377] Logged in from keytab as kudu/127.11.8.65@KRBTEST.COM (short username kudu)
I20260806 14:58:26.578161 15601 webserver.cc:460] Webserver started at http://127.11.8.65:44355/ using document root <none> and password file <none>
I20260806 14:58:26.578459 15601 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:26.578543 15601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:26.578683 15601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:26.579751 15601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/data/instance:
uuid: "883cb14c1a2f47eebf17061cc5099728"
format_stamp: "Formatted at 2026-08-06 14:58:26 on dist-test-slave-68xk"
server_key: "e470b0509e5e265378551f7b72c5bf1c"
server_key_iv: "6a6684e312f6c63fbc1711c392235aa2"
server_key_version: "encryptionkey@0"
I20260806 14:58:26.580128 15601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance:
uuid: "883cb14c1a2f47eebf17061cc5099728"
format_stamp: "Formatted at 2026-08-06 14:58:26 on dist-test-slave-68xk"
server_key: "e470b0509e5e265378551f7b72c5bf1c"
server_key_iv: "6a6684e312f6c63fbc1711c392235aa2"
server_key_version: "encryptionkey@0"
I20260806 14:58:26.581549 15601 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 14:58:26.582465 15622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:26.582677 15601 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:26.582787 15601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "883cb14c1a2f47eebf17061cc5099728"
format_stamp: "Formatted at 2026-08-06 14:58:26 on dist-test-slave-68xk"
server_key: "e470b0509e5e265378551f7b72c5bf1c"
server_key_iv: "6a6684e312f6c63fbc1711c392235aa2"
server_key_version: "encryptionkey@0"
I20260806 14:58:26.582862 15601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:26.590860 15601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:26.592729 15601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:26.592890 15601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:26.593112 15601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:26.593433 15601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:26.593489 15601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:26.593530 15601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:26.593559 15601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:26.599834 15601 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:46525
I20260806 14:58:26.599896 15773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:46525 every 8 connection(s)
I20260806 14:58:26.600232 15601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
I20260806 14:58:26.600800 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 15601
I20260806 14:58:26.600889 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance
I20260806 14:58:26.601107 11297 external_mini_cluster.cc:1428] Setting key ce5a9a7ab4740c79527f355158ef9536
Aug 06 14:58:26 dist-test-slave-68xk krb5kdc[15450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028306, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.8.66@KRBTEST.COM; defaulting to no policy
I20260806 14:58:26.611932 15565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:26.601089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:45819 (local address 127.11.8.126:40517)
0806 14:58:26.601163 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:26.601166 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:26.601532 (+ 366us) server_negotiation.cc:407] Connection header received
0806 14:58:26.602225 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:26.602226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:26.602272 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:26.602326 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:26.603665 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:26.604221 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:26.605366 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:26.605606 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:26.605697 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:26.608493 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:26.608495 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:26.608495 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:26.608503 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:26.610386 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:26.610888 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:26.610889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:26.610890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:26.610959 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:26.611331 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:26.611332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:26.611333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:26.611450 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:26.611518 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:26.611788 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 14:58:26.611865 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 14:58:26.612160 15776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:26.601075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40517 (local address 127.11.8.65:45819)
0806 14:58:26.601386 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:26.601435 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:26.602103 (+ 668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:26.602463 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:26.602469 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:26.603248 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:26.603550 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:26.603557 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:26.604333 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:26.604334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:26.605255 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:26.605258 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:26.605706 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:26.605707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:26.605853 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:26.606502 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:26.606511 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:26.608373 (+ 1862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:26.610531 (+ 2158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:26.610538 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:26.610540 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:26.610799 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:26.611070 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:26.611071 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:26.611072 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:26.611234 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:26.611527 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:26.611535 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:26.611660 (+ 125us) client_negotiation.cc:769] Sending connection context
0806 14:58:26.611750 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 14:58:26.611877 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":93,"threads_started":1}
I20260806 14:58:26.612742 15774 heartbeater.cc:347] Connected to a master server at 127.11.8.126:40517
I20260806 14:58:26.612856 15774 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:26.613040 15774 heartbeater.cc:511] Master 127.11.8.126:40517 requested a full tablet report, sending...
I20260806 14:58:26.613783 15515 ts_manager.cc:194] Registered new tserver with Master: 883cb14c1a2f47eebf17061cc5099728 (127.11.8.65:46525)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.66@KRBTEST.COM" created.
I20260806 14:58:26.614645 15515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.65@KRBTEST.COM'} at 127.11.8.65:45819
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
I20260806 14:58:26.629951 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:0
--local_ip_for_outbound_sockets=127.11.8.66
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:40517
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:42199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:26.713969 15779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:26.714174 15779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:26.714198 15779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:26.715472 15779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:26.715518 15779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:26.715545 15779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 14:58:26.716933 15779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:42199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:40517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:26.717123 15779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:26.717290 15779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:26.722250 15788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:26.722278 15790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:26.722276 15786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:26.722465 15779 server_base.cc:1034] running on GCE node
I20260806 14:58:26.722759 15779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:26.723030 15779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:26.724174 15779 hybrid_clock.cc:648] HybridClock initialized: now 1786028306724163 us; error 28 us; skew 500 ppm
Aug 06 14:58:26 dist-test-slave-68xk krb5kdc[15450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028306, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:26.726821 15779 init.cc:377] Logged in from keytab as kudu/127.11.8.66@KRBTEST.COM (short username kudu)
I20260806 14:58:26.727210 15779 webserver.cc:460] Webserver started at http://127.11.8.66:35509/ using document root <none> and password file <none>
I20260806 14:58:26.727422 15779 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:26.727464 15779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:26.727557 15779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:26.728540 15779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/data/instance:
uuid: "cdfa1bea80f743959d77a74b83f85850"
format_stamp: "Formatted at 2026-08-06 14:58:26 on dist-test-slave-68xk"
server_key: "33c44887f80ef9815763c5fec497ddb1"
server_key_iv: "433c58749ec0f8c571067597fa8d9dbf"
server_key_version: "encryptionkey@0"
I20260806 14:58:26.728843 15779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance:
uuid: "cdfa1bea80f743959d77a74b83f85850"
format_stamp: "Formatted at 2026-08-06 14:58:26 on dist-test-slave-68xk"
server_key: "33c44887f80ef9815763c5fec497ddb1"
server_key_iv: "433c58749ec0f8c571067597fa8d9dbf"
server_key_version: "encryptionkey@0"
I20260806 14:58:26.730100 15779 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 14:58:26.730908 15799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:26.731096 15779 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:26.731197 15779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "cdfa1bea80f743959d77a74b83f85850"
format_stamp: "Formatted at 2026-08-06 14:58:26 on dist-test-slave-68xk"
server_key: "33c44887f80ef9815763c5fec497ddb1"
server_key_iv: "433c58749ec0f8c571067597fa8d9dbf"
server_key_version: "encryptionkey@0"
I20260806 14:58:26.731302 15779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:26.750707 15779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:26.752699 15779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:26.752861 15779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:26.753093 15779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:26.753408 15779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:26.753464 15779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:26.753505 15779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:26.753532 15779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:26.759361 15779 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:33963
I20260806 14:58:26.759403 15952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:33963 every 8 connection(s)
I20260806 14:58:26.759749 15779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
I20260806 14:58:26.765458 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 15779
I20260806 14:58:26.765575 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance
I20260806 14:58:26.765766 11297 external_mini_cluster.cc:1428] Setting key 19ee62add224d3ab7d49efd4eebdf79b
Aug 06 14:58:26 dist-test-slave-68xk krb5kdc[15450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028306, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:26.770093 15565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:26.760487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:59737 (local address 127.11.8.126:40517)
0806 14:58:26.760559 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:26.760561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:26.760748 (+ 187us) server_negotiation.cc:407] Connection header received
0806 14:58:26.761283 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:26.761285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:26.761298 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:26.761349 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:26.762402 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:26.762934 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:26.763979 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:26.764222 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:26.764302 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:26.766808 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:26.766810 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:26.766810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:26.766818 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:26.768444 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:26.768982 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:26.768984 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:26.768984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:26.769050 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:26.769345 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:26.769346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:26.769346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:26.769451 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:26.769506 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:26.769862 (+ 356us) server_negotiation.cc:299] Negotiation successful
0806 14:58:26.769953 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 14:58:26.770134 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:26.760462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40517 (local address 127.11.8.66:59737)
0806 14:58:26.760625 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:26.760660 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:26.761166 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:26.761437 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:26.761441 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:26.762012 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:26.762295 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:26.762301 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:26.763039 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:26.763040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:26.763878 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:26.763881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:26.764308 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:26.764310 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:26.764444 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:26.764900 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:26.764909 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:26.766699 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:26.768542 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:26.768550 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:26.768552 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:26.768898 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:26.769132 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:26.769133 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:26.769133 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:26.769275 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:26.769519 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:26.769525 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:26.769696 (+ 171us) client_negotiation.cc:769] Sending connection context
0806 14:58:26.769817 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 14:58:26.769971 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":78,"threads_started":1}
I20260806 14:58:26.770668 15953 heartbeater.cc:347] Connected to a master server at 127.11.8.126:40517
I20260806 14:58:26.770788 15953 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:26.770984 15953 heartbeater.cc:511] Master 127.11.8.126:40517 requested a full tablet report, sending...
I20260806 14:58:26.771355 15515 ts_manager.cc:194] Registered new tserver with Master: cdfa1bea80f743959d77a74b83f85850 (127.11.8.66:33963)
I20260806 14:58:26.771720 15515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.66@KRBTEST.COM'} at 127.11.8.66:59737
WARNING: no policy specified for kudu/127.11.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
I20260806 14:58:26.794114 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:0
--local_ip_for_outbound_sockets=127.11.8.67
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:40517
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:42199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:26.876276 15959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:26.876484 15959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:26.876507 15959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:26.877691 15959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:26.877734 15959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:26.877761 15959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 14:58:26.879355 15959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:42199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:40517
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:26.879608 15959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:26.879844 15959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:26.885095 15971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:26.885047 15969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:26.885047 15968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:26.885192 15959 server_base.cc:1034] running on GCE node
I20260806 14:58:26.885509 15959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:26.885717 15959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:26.886899 15959 hybrid_clock.cc:648] HybridClock initialized: now 1786028306886900 us; error 44 us; skew 500 ppm
Aug 06 14:58:26 dist-test-slave-68xk krb5kdc[15450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028306, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:26.889650 15959 init.cc:377] Logged in from keytab as kudu/127.11.8.67@KRBTEST.COM (short username kudu)
I20260806 14:58:26.890204 15959 webserver.cc:460] Webserver started at http://127.11.8.67:41107/ using document root <none> and password file <none>
I20260806 14:58:26.890509 15959 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:26.890581 15959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:26.890750 15959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:26.891817 15959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/data/instance:
uuid: "38b6d395e67444059fe01bdc423e0c3b"
format_stamp: "Formatted at 2026-08-06 14:58:26 on dist-test-slave-68xk"
server_key: "7d753d339af4cbf60f3131683abe6fea"
server_key_iv: "25bdc7e310652138fbf6b18f2d956b3e"
server_key_version: "encryptionkey@0"
I20260806 14:58:26.892202 15959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance:
uuid: "38b6d395e67444059fe01bdc423e0c3b"
format_stamp: "Formatted at 2026-08-06 14:58:26 on dist-test-slave-68xk"
server_key: "7d753d339af4cbf60f3131683abe6fea"
server_key_iv: "25bdc7e310652138fbf6b18f2d956b3e"
server_key_version: "encryptionkey@0"
I20260806 14:58:26.893615 15959 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 14:58:26.894544 15981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:26.894780 15959 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:26.894886 15959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "38b6d395e67444059fe01bdc423e0c3b"
format_stamp: "Formatted at 2026-08-06 14:58:26 on dist-test-slave-68xk"
server_key: "7d753d339af4cbf60f3131683abe6fea"
server_key_iv: "25bdc7e310652138fbf6b18f2d956b3e"
server_key_version: "encryptionkey@0"
I20260806 14:58:26.894973 15959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:26.903388 15959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:26.905773 15959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:26.905941 15959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:26.906170 15959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:26.906608 15959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:26.906674 15959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:26.906716 15959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:26.906744 15959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:26.913010 15959 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:42381
I20260806 14:58:26.913055 16127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:42381 every 8 connection(s)
I20260806 14:58:26.913388 15959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
I20260806 14:58:26.919291 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 15959
I20260806 14:58:26.919363 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance
I20260806 14:58:26.919556 11297 external_mini_cluster.cc:1428] Setting key 575f1719b0dee1dc251b1b42109445c0
Aug 06 14:58:26 dist-test-slave-68xk krb5kdc[15450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028306, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:26.923167 15565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:26.914150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:49781 (local address 127.11.8.126:40517)
0806 14:58:26.914197 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:26.914200 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:26.914476 (+ 276us) server_negotiation.cc:407] Connection header received
0806 14:58:26.914962 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:26.914963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:26.914975 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:26.915044 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:26.916017 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:26.916524 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:26.917579 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:26.917777 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:26.917837 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:26.920352 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:26.920353 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:26.920354 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:26.920363 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:26.921698 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:26.922155 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:26.922156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:26.922156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:26.922209 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:26.922501 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:26.922502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:26.922502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:26.922621 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:26.922694 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:26.922982 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 14:58:26.923080 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 14:58:26.923327 16130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:26.914091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40517 (local address 127.11.8.67:49781)
0806 14:58:26.914266 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:26.914311 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:26.914867 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:26.915100 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:26.915103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:26.915684 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:26.915905 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:26.915910 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:26.916654 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:26.916656 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:26.917476 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:26.917478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:26.917857 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:26.917858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:26.917973 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:26.918536 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:26.918545 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:26.920273 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:26.921796 (+ 1523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:26.921803 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:26.921805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:26.922038 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:26.922284 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:26.922285 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:26.922285 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:26.922401 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:26.922698 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:26.922704 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:26.922841 (+ 137us) client_negotiation.cc:769] Sending connection context
0806 14:58:26.922942 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 14:58:26.923098 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":101,"threads_started":1}
I20260806 14:58:26.923834 16128 heartbeater.cc:347] Connected to a master server at 127.11.8.126:40517
I20260806 14:58:26.923933 16128 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:26.924072 16128 heartbeater.cc:511] Master 127.11.8.126:40517 requested a full tablet report, sending...
I20260806 14:58:26.924441 15515 ts_manager.cc:194] Registered new tserver with Master: 38b6d395e67444059fe01bdc423e0c3b (127.11.8.67:42381)
I20260806 14:58:26.924811 15515 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.67@KRBTEST.COM'} at 127.11.8.67:49781
I20260806 14:58:26.931648 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 14:58:26 dist-test-slave-68xk krb5kdc[15450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028306, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 14:58:26 dist-test-slave-68xk krb5kdc[15450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028306, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:26.955613 15565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:26.948303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55534 (local address 127.11.8.126:40517)
0806 14:58:26.948380 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:26.948382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:26.948521 (+ 139us) server_negotiation.cc:407] Connection header received
0806 14:58:26.948588 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:26.948589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:26.948599 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:26.948651 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:26.949232 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:26.949729 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:26.950546 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:26.950735 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:26.950793 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:26.952978 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:26.952980 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:26.952980 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:26.952987 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:26.954528 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:26.954878 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:26.954879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:26.954879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:26.954946 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:26.955134 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:26.955134 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:26.955135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:26.955283 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:26.955359 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:26.955452 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 14:58:26.955510 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260806 14:58:26.955912 15515 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:55534
I20260806 14:58:26.956391 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 15601
I20260806 14:58:26.970266 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 15779
I20260806 14:58:26.986387 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 15959
I20260806 14:58:27.006732 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 15469
2026-08-06T14:58:27Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2143 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 14:58:27 dist-test-slave-68xk krb5kdc[16140](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 14:58:27 dist-test-slave-68xk krb5kdc[16140](info): set up 2 sockets
Aug 06 14:58:27 dist-test-slave-68xk krb5kdc[16140](info): commencing operation
krb5kdc: starting...
W20260806 14:58:28.242568 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 14:58:28 dist-test-slave-68xk krb5kdc[16140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T14:58:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T14:58:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 14:58:28.368911 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:32827
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:38343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:32827
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:28.451009 16156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:28.451303 16156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:28.451340 16156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:28.452775 16156 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 14:58:28.452817 16156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:28.452832 16156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:28.452844 16156 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 14:58:28.452883 16156 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 14:58:28.454385 16156 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:38343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:32827
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:32827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16156
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:28.454651 16156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:28.454907 16156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:28.459916 16165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:28.460021 16156 server_base.cc:1034] running on GCE node
W20260806 14:58:28.459916 16162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:28.459990 16163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:28.460372 16156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:28.460628 16156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:28.461771 16156 hybrid_clock.cc:648] HybridClock initialized: now 1786028308461766 us; error 39 us; skew 500 ppm
Aug 06 14:58:28 dist-test-slave-68xk krb5kdc[16140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028308, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:28.464486 16156 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 14:58:28.464963 16156 webserver.cc:460] Webserver started at http://127.11.8.126:38725/ using document root <none> and password file <none>
I20260806 14:58:28.465229 16156 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:28.465284 16156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:28.465418 16156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:28.466614 16156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "639958b9186441ed82878cf7ed705f89"
format_stamp: "Formatted at 2026-08-06 14:58:28 on dist-test-slave-68xk"
server_key: "01f43fcde527a81c1b49d4963cb3684c"
server_key_iv: "dc7535dd9a5573563d7d10680fe88d2e"
server_key_version: "encryptionkey@0"
I20260806 14:58:28.466980 16156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "639958b9186441ed82878cf7ed705f89"
format_stamp: "Formatted at 2026-08-06 14:58:28 on dist-test-slave-68xk"
server_key: "01f43fcde527a81c1b49d4963cb3684c"
server_key_iv: "dc7535dd9a5573563d7d10680fe88d2e"
server_key_version: "encryptionkey@0"
I20260806 14:58:28.468367 16156 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:28.469125 16178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:28.469348 16156 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:28.469444 16156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "639958b9186441ed82878cf7ed705f89"
format_stamp: "Formatted at 2026-08-06 14:58:28 on dist-test-slave-68xk"
server_key: "01f43fcde527a81c1b49d4963cb3684c"
server_key_iv: "dc7535dd9a5573563d7d10680fe88d2e"
server_key_version: "encryptionkey@0"
I20260806 14:58:28.469515 16156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:28.500495 16156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:28.502971 16156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:28.503149 16156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:28.507392 16156 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:32827
I20260806 14:58:28.507674 16250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:32827 every 8 connection(s)
I20260806 14:58:28.507774 16156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 14:58:28.508471 16251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:28.510803 16251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 639958b9186441ed82878cf7ed705f89: Bootstrap starting.
I20260806 14:58:28.511348 16251 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 639958b9186441ed82878cf7ed705f89: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:28.511585 16251 log.cc:824] T 00000000000000000000000000000000 P 639958b9186441ed82878cf7ed705f89: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:28.512238 16251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 639958b9186441ed82878cf7ed705f89: No bootstrap required, opened a new log
I20260806 14:58:28.513751 16251 raft_consensus.cc:348] T 00000000000000000000000000000000 P 639958b9186441ed82878cf7ed705f89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "639958b9186441ed82878cf7ed705f89" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 32827 } }
I20260806 14:58:28.513883 16251 raft_consensus.cc:374] T 00000000000000000000000000000000 P 639958b9186441ed82878cf7ed705f89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:28.513906 16251 raft_consensus.cc:729] T 00000000000000000000000000000000 P 639958b9186441ed82878cf7ed705f89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 639958b9186441ed82878cf7ed705f89, State: Initialized, Role: FOLLOWER
I20260806 14:58:28.514004 16251 consensus_queue.cc:260] T 00000000000000000000000000000000 P 639958b9186441ed82878cf7ed705f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "639958b9186441ed82878cf7ed705f89" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 32827 } }
I20260806 14:58:28.514056 16251 raft_consensus.cc:388] T 00000000000000000000000000000000 P 639958b9186441ed82878cf7ed705f89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 14:58:28.514087 16251 raft_consensus.cc:482] T 00000000000000000000000000000000 P 639958b9186441ed82878cf7ed705f89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 14:58:28.514118 16251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 639958b9186441ed82878cf7ed705f89 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:28.514385 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 16156
I20260806 14:58:28.514511 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 14:58:28.514736 11297 external_mini_cluster.cc:1428] Setting key 2bde15e7cf0d82363163febc16994266
I20260806 14:58:28.515316 16251 raft_consensus.cc:504] T 00000000000000000000000000000000 P 639958b9186441ed82878cf7ed705f89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "639958b9186441ed82878cf7ed705f89" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 32827 } }
I20260806 14:58:28.515439 16251 leader_election.cc:302] T 00000000000000000000000000000000 P 639958b9186441ed82878cf7ed705f89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 639958b9186441ed82878cf7ed705f89; no voters:
I20260806 14:58:28.515641 16251 leader_election.cc:288] T 00000000000000000000000000000000 P 639958b9186441ed82878cf7ed705f89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 14:58:28.515681 16257 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 639958b9186441ed82878cf7ed705f89 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:28.515818 16257 raft_consensus.cc:686] T 00000000000000000000000000000000 P 639958b9186441ed82878cf7ed705f89 [term 1 LEADER]: Becoming Leader. State: Replica: 639958b9186441ed82878cf7ed705f89, State: Running, Role: LEADER
I20260806 14:58:28.515923 16251 sys_catalog.cc:564] T 00000000000000000000000000000000 P 639958b9186441ed82878cf7ed705f89 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 14:58:28.515910 16257 consensus_queue.cc:237] T 00000000000000000000000000000000 P 639958b9186441ed82878cf7ed705f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "639958b9186441ed82878cf7ed705f89" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 32827 } }
I20260806 14:58:28.516310 16258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 639958b9186441ed82878cf7ed705f89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "639958b9186441ed82878cf7ed705f89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "639958b9186441ed82878cf7ed705f89" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 32827 } } }
I20260806 14:58:28.516479 16258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 639958b9186441ed82878cf7ed705f89 [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:28.516911 16259 sys_catalog.cc:455] T 00000000000000000000000000000000 P 639958b9186441ed82878cf7ed705f89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 639958b9186441ed82878cf7ed705f89. Latest consensus state: current_term: 1 leader_uuid: "639958b9186441ed82878cf7ed705f89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "639958b9186441ed82878cf7ed705f89" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 32827 } } }
I20260806 14:58:28.517000 16259 sys_catalog.cc:458] T 00000000000000000000000000000000 P 639958b9186441ed82878cf7ed705f89 [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:28.517088 16267 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 14:58:28.517632 16267 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 14:58:28.519359 16267 catalog_manager.cc:1347] Generated new cluster ID: 98334f85c52749ebb083560fbe970b92
I20260806 14:58:28.519421 16267 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 14:58:28 dist-test-slave-68xk krb5kdc[16140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028308, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:28.525640 16256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:28.515329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51178 (local address 127.11.8.126:32827)
0806 14:58:28.515536 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:28.515546 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:28.515588 (+ 42us) server_negotiation.cc:407] Connection header received
0806 14:58:28.515713 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:28.515721 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:28.515807 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:28.516805 (+ 998us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:28.516931 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:28.518322 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:28.519308 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:28.519780 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:28.519901 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:28.522268 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:28.522271 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:28.522272 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:28.522284 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:28.524028 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:28.524414 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:28.524417 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:28.524418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:28.524526 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:28.524749 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:28.524750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:28.524750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:28.524989 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:28.525216 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:28.525289 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 14:58:28.525432 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":99,"threads_started":1}
I20260806 14:58:28.530541 16267 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 14:58:28.531222 16267 catalog_manager.cc:1504] Loading token signing keys...
I20260806 14:58:28.535801 16267 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 639958b9186441ed82878cf7ed705f89: Generated new TSK 0
I20260806 14:58:28.536003 16267 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
I20260806 14:58:28.570654 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:0
--local_ip_for_outbound_sockets=127.11.8.65
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:32827
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:38343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:28.674060 16282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:28.674273 16282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:28.674301 16282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:28.675625 16282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:28.675671 16282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:28.675701 16282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 14:58:28.677472 16282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:38343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:32827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:28.677755 16282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:28.678017 16282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:28.683446 16287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:28.683395 16288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:28.683403 16290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:28.683779 16282 server_base.cc:1034] running on GCE node
I20260806 14:58:28.684015 16282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:28.684257 16282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:28.685397 16282 hybrid_clock.cc:648] HybridClock initialized: now 1786028308685375 us; error 45 us; skew 500 ppm
Aug 06 14:58:28 dist-test-slave-68xk krb5kdc[16140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028308, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:28.687995 16282 init.cc:377] Logged in from keytab as kudu/127.11.8.65@KRBTEST.COM (short username kudu)
I20260806 14:58:28.688421 16282 webserver.cc:460] Webserver started at http://127.11.8.65:42613/ using document root <none> and password file <none>
I20260806 14:58:28.688661 16282 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:28.688705 16282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:28.688877 16282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:28.689922 16282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-0/data/instance:
uuid: "321a16b53cd649bcbcee156d5892efff"
format_stamp: "Formatted at 2026-08-06 14:58:28 on dist-test-slave-68xk"
server_key: "d6a80d1801cdddfd4476069b66b66088"
server_key_iv: "a451a61bd752d44616e12d784910bca2"
server_key_version: "encryptionkey@0"
I20260806 14:58:28.690274 16282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance:
uuid: "321a16b53cd649bcbcee156d5892efff"
format_stamp: "Formatted at 2026-08-06 14:58:28 on dist-test-slave-68xk"
server_key: "d6a80d1801cdddfd4476069b66b66088"
server_key_iv: "a451a61bd752d44616e12d784910bca2"
server_key_version: "encryptionkey@0"
I20260806 14:58:28.691677 16282 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 14:58:28.692490 16299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:28.692713 16282 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 14:58:28.692806 16282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "321a16b53cd649bcbcee156d5892efff"
format_stamp: "Formatted at 2026-08-06 14:58:28 on dist-test-slave-68xk"
server_key: "d6a80d1801cdddfd4476069b66b66088"
server_key_iv: "a451a61bd752d44616e12d784910bca2"
server_key_version: "encryptionkey@0"
I20260806 14:58:28.692900 16282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:28.705669 16282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:28.707675 16282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:28.707844 16282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:28.708071 16282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:28.708381 16282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:28.708438 16282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:28.708472 16282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:28.708499 16282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:28.714527 16282 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:40731
I20260806 14:58:28.714589 16453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:40731 every 8 connection(s)
I20260806 14:58:28.714933 16282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
I20260806 14:58:28.717521 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 16282
I20260806 14:58:28.717666 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance
I20260806 14:58:28.717865 11297 external_mini_cluster.cc:1428] Setting key fc8227322be7f7d76e5c2cb14c9c4aa2
Aug 06 14:58:28 dist-test-slave-68xk krb5kdc[16140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028308, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:28.726395 16256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:28.715744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:59741 (local address 127.11.8.126:32827)
0806 14:58:28.715842 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:28.715844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:28.716042 (+ 198us) server_negotiation.cc:407] Connection header received
0806 14:58:28.717166 (+ 1124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:28.717168 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:28.717184 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:28.717238 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:28.718105 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:28.718836 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:28.720379 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:28.720719 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:28.720800 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:28.723448 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:28.723450 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:28.723450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:28.723459 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:28.724858 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:28.725401 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:28.725402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:28.725402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:28.725452 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:28.725762 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:28.725762 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:28.725763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:28.725861 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:28.725911 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:28.726260 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 14:58:28.726328 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 14:58:28.726672 16457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:28.715693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:32827 (local address 127.11.8.65:59741)
0806 14:58:28.715907 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:28.715952 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:28.717043 (+ 1091us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:28.717310 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:28.717314 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:28.717718 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:28.717997 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:28.718003 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:28.718941 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:28.718942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:28.720271 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:28.720275 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:28.720826 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:28.720827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:28.720937 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:28.721525 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:28.721534 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:28.723353 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:28.724945 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:28.724953 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:28.724955 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:28.725324 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:28.725541 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:28.725542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:28.725542 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:28.725691 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:28.725928 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:28.725934 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:28.726116 (+ 182us) client_negotiation.cc:769] Sending connection context
0806 14:58:28.726237 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 14:58:28.726429 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":81,"threads_started":1}
I20260806 14:58:28.727231 16454 heartbeater.cc:347] Connected to a master server at 127.11.8.126:32827
I20260806 14:58:28.727339 16454 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:28.727540 16454 heartbeater.cc:511] Master 127.11.8.126:32827 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.11.8.66@KRBTEST.COM; defaulting to no policy
I20260806 14:58:28.728214 16207 ts_manager.cc:194] Registered new tserver with Master: 321a16b53cd649bcbcee156d5892efff (127.11.8.65:40731)
I20260806 14:58:28.728886 16207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.65@KRBTEST.COM'} at 127.11.8.65:59741
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
I20260806 14:58:28.747309 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:0
--local_ip_for_outbound_sockets=127.11.8.66
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:32827
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:38343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:28.831744 16461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:28.832038 16461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:28.832078 16461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:28.833186 16461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:28.833230 16461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:28.833256 16461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 14:58:28.834728 16461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:38343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:32827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:28.834913 16461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:28.835099 16461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:28.840313 16468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:28.840340 16469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:28.840518 16461 server_base.cc:1034] running on GCE node
W20260806 14:58:28.840371 16474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:28.840849 16461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:28.841106 16461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:28.842299 16461 hybrid_clock.cc:648] HybridClock initialized: now 1786028308842284 us; error 53 us; skew 500 ppm
Aug 06 14:58:28 dist-test-slave-68xk krb5kdc[16140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028308, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:28.845257 16461 init.cc:377] Logged in from keytab as kudu/127.11.8.66@KRBTEST.COM (short username kudu)
I20260806 14:58:28.845820 16461 webserver.cc:460] Webserver started at http://127.11.8.66:38151/ using document root <none> and password file <none>
I20260806 14:58:28.846031 16461 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:28.846076 16461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:28.846174 16461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:28.847370 16461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-1/data/instance:
uuid: "60ccfccad2354560b2c1f31aec005e55"
format_stamp: "Formatted at 2026-08-06 14:58:28 on dist-test-slave-68xk"
server_key: "70ad0c313650f84c6fb169049a858d42"
server_key_iv: "b0e7b10999101ac8d144f49971687551"
server_key_version: "encryptionkey@0"
I20260806 14:58:28.847710 16461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance:
uuid: "60ccfccad2354560b2c1f31aec005e55"
format_stamp: "Formatted at 2026-08-06 14:58:28 on dist-test-slave-68xk"
server_key: "70ad0c313650f84c6fb169049a858d42"
server_key_iv: "b0e7b10999101ac8d144f49971687551"
server_key_version: "encryptionkey@0"
I20260806 14:58:28.849131 16461 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 14:58:28.849888 16483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:28.850080 16461 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:28.850193 16461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "60ccfccad2354560b2c1f31aec005e55"
format_stamp: "Formatted at 2026-08-06 14:58:28 on dist-test-slave-68xk"
server_key: "70ad0c313650f84c6fb169049a858d42"
server_key_iv: "b0e7b10999101ac8d144f49971687551"
server_key_version: "encryptionkey@0"
I20260806 14:58:28.850252 16461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:28.873050 16461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:28.875128 16461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:28.875252 16461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:28.875460 16461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:28.875741 16461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:28.875773 16461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:28.875798 16461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:28.875818 16461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:28.882220 16461 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:45961
I20260806 14:58:28.882318 16634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:45961 every 8 connection(s)
I20260806 14:58:28.882669 16461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
I20260806 14:58:28.882957 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 16461
I20260806 14:58:28.883122 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance
I20260806 14:58:28.883364 11297 external_mini_cluster.cc:1428] Setting key 5a87261b1c7ad266459b432eb0afa768
Aug 06 14:58:28 dist-test-slave-68xk krb5kdc[16140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028308, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.8.67@KRBTEST.COM; defaulting to no policy
I20260806 14:58:28.894243 16256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:28.883612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:46503 (local address 127.11.8.126:32827)
0806 14:58:28.883687 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:28.883689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:28.883969 (+ 280us) server_negotiation.cc:407] Connection header received
0806 14:58:28.885369 (+ 1400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:28.885370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:28.885380 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:28.885435 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:28.886382 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:28.887294 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:28.888349 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:28.888594 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:28.888691 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:28.891278 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:28.891280 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:28.891281 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:28.891290 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:28.892732 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:28.893182 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:28.893183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:28.893184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:28.893284 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:28.893548 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:28.893549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:28.893550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:28.893670 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:28.893751 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:28.894043 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 14:58:28.894133 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 14:58:28.894446 16638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:28.883617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:32827 (local address 127.11.8.66:46503)
0806 14:58:28.883845 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:28.883885 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:28.885238 (+ 1353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:28.885534 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:28.885539 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:28.885958 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:28.886262 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:28.886269 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:28.887423 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:28.887424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:28.888255 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:28.888258 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:28.888710 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:28.888712 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:28.888852 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:28.889458 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:28.889468 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:28.891169 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:28.892826 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:28.892834 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:28.892857 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:28.893089 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:28.893356 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:28.893357 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:28.893358 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:28.893478 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:28.893753 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:28.893760 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:28.893903 (+ 143us) client_negotiation.cc:769] Sending connection context
0806 14:58:28.894009 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 14:58:28.894173 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":94,"threads_started":1}
I20260806 14:58:28.894989 16636 heartbeater.cc:347] Connected to a master server at 127.11.8.126:32827
I20260806 14:58:28.895124 16636 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:28.895362 16636 heartbeater.cc:511] Master 127.11.8.126:32827 requested a full tablet report, sending...
I20260806 14:58:28.895804 16207 ts_manager.cc:194] Registered new tserver with Master: 60ccfccad2354560b2c1f31aec005e55 (127.11.8.66:45961)
I20260806 14:58:28.896188 16207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.66@KRBTEST.COM'} at 127.11.8.66:46503
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
I20260806 14:58:28.913241 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:0
--local_ip_for_outbound_sockets=127.11.8.67
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:32827
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:38343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:28.996522 16642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:28.996721 16642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:28.996742 16642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:28.997852 16642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:28.997890 16642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:28.997917 16642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 14:58:28.999500 16642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:38343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:32827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:28.999697 16642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:28.999871 16642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:29.004936 16649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:29.004958 16652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:29.004997 16642 server_base.cc:1034] running on GCE node
W20260806 14:58:29.005127 16648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:29.005420 16642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:29.005713 16642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:29.006891 16642 hybrid_clock.cc:648] HybridClock initialized: now 1786028309006877 us; error 29 us; skew 500 ppm
Aug 06 14:58:29 dist-test-slave-68xk krb5kdc[16140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028309, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:29.010613 16642 init.cc:377] Logged in from keytab as kudu/127.11.8.67@KRBTEST.COM (short username kudu)
I20260806 14:58:29.011209 16642 webserver.cc:460] Webserver started at http://127.11.8.67:38021/ using document root <none> and password file <none>
I20260806 14:58:29.011535 16642 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:29.011603 16642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:29.011749 16642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:29.013149 16642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-2/data/instance:
uuid: "024f1775ef3544e788fd7ecab897503f"
format_stamp: "Formatted at 2026-08-06 14:58:29 on dist-test-slave-68xk"
server_key: "9ac88834348f7c3cf7711caddc96a0ff"
server_key_iv: "563f8262a35c9a48d076dff53ba95140"
server_key_version: "encryptionkey@0"
I20260806 14:58:29.013572 16642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance:
uuid: "024f1775ef3544e788fd7ecab897503f"
format_stamp: "Formatted at 2026-08-06 14:58:29 on dist-test-slave-68xk"
server_key: "9ac88834348f7c3cf7711caddc96a0ff"
server_key_iv: "563f8262a35c9a48d076dff53ba95140"
server_key_version: "encryptionkey@0"
I20260806 14:58:29.015344 16642 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260806 14:58:29.016089 16663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:29.016248 16642 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:29.016323 16642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "024f1775ef3544e788fd7ecab897503f"
format_stamp: "Formatted at 2026-08-06 14:58:29 on dist-test-slave-68xk"
server_key: "9ac88834348f7c3cf7711caddc96a0ff"
server_key_iv: "563f8262a35c9a48d076dff53ba95140"
server_key_version: "encryptionkey@0"
I20260806 14:58:29.016417 16642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:29.037701 16642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:29.039776 16642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:29.039939 16642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:29.040174 16642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:29.040496 16642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:29.040555 16642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:29.040606 16642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:29.040634 16642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:29.046566 16642 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:32841
I20260806 14:58:29.046620 16815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:32841 every 8 connection(s)
I20260806 14:58:29.046980 16642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
I20260806 14:58:29.049036 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 16642
I20260806 14:58:29.049203 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance
I20260806 14:58:29.049408 11297 external_mini_cluster.cc:1428] Setting key b0e2a21e1ea55616dd5b3687f6bc8ad5
Aug 06 14:58:29 dist-test-slave-68xk krb5kdc[16140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028309, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:29.056740 16256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:29.047724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:54191 (local address 127.11.8.126:32827)
0806 14:58:29.047785 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:29.047787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:29.047975 (+ 188us) server_negotiation.cc:407] Connection header received
0806 14:58:29.048916 (+ 941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:29.048917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:29.048928 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:29.048985 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:29.049869 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:29.050599 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:29.051554 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:29.051747 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:29.051798 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:29.053882 (+ 2084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:29.053883 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:29.053884 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:29.053891 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:29.055488 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:29.055852 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:29.055853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:29.055854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:29.055906 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:29.056114 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:29.056115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:29.056115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:29.056212 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:29.056285 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:29.056569 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 14:58:29.056659 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 14:58:29.056859 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:29.047675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:32827 (local address 127.11.8.67:54191)
0806 14:58:29.047840 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:29.047881 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:29.048786 (+ 905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:29.049064 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:29.049069 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:29.049495 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:29.049774 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:29.049781 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:29.050686 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:29.050687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:29.051483 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:29.051485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:29.051825 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:29.051826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:29.051927 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:29.052393 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:29.052402 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:29.053795 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:29.055567 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:29.055573 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:29.055575 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:29.055786 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:29.055966 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:29.055967 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:29.055967 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:29.056062 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:29.056297 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:29.056303 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:29.056422 (+ 119us) client_negotiation.cc:769] Sending connection context
0806 14:58:29.056520 (+ 98us) client_negotiation.cc:240] Negotiation successful
0806 14:58:29.056666 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":95,"threads_started":1}
I20260806 14:58:29.057365 16816 heartbeater.cc:347] Connected to a master server at 127.11.8.126:32827
I20260806 14:58:29.057484 16816 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:29.057660 16816 heartbeater.cc:511] Master 127.11.8.126:32827 requested a full tablet report, sending...
I20260806 14:58:29.058135 16207 ts_manager.cc:194] Registered new tserver with Master: 024f1775ef3544e788fd7ecab897503f (127.11.8.67:32841)
I20260806 14:58:29.058600 16207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.67@KRBTEST.COM'} at 127.11.8.67:54191
I20260806 14:58:29.061049 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 14:58:29 dist-test-slave-68xk krb5kdc[16140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 14:58:29 dist-test-slave-68xk krb5kdc[16140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.65@KRBTEST.COM
I20260806 14:58:29.085878 16828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:29.077989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39598 (local address 127.11.8.65:40731)
0806 14:58:29.078174 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:29.078181 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:29.078196 (+ 15us) server_negotiation.cc:407] Connection header received
0806 14:58:29.078238 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:29.078240 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:29.078256 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:29.078339 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:29.078980 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:29.079776 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:29.080589 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:29.080937 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:29.080997 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:29.083321 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:29.083330 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:29.083330 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:29.083337 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:29.084787 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:29.085152 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:29.085155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:29.085155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:29.085224 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:29.085440 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:29.085440 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:29.085441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:29.085568 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:29.085683 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:29.085741 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 14:58:29.085803 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":106,"threads_started":1}
Aug 06 14:58:29 dist-test-slave-68xk krb5kdc[16140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:29.094094 16256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:29.087330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51188 (local address 127.11.8.126:32827)
0806 14:58:29.087400 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:29.087402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:29.087411 (+ 9us) server_negotiation.cc:407] Connection header received
0806 14:58:29.087459 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:29.087460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:29.087470 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:29.087515 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:29.087934 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:29.088417 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:29.089091 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:29.089286 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:29.089351 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:29.091571 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:29.091573 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:29.091573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:29.091581 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:29.092850 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:29.093223 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:29.093225 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:29.093225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:29.093287 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:29.093503 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:29.093504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:29.093505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:29.093661 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:29.093783 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:29.093896 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 14:58:29.093976 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 14:58:29.094339 16207 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51188
I20260806 14:58:29.094734 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 16282
I20260806 14:58:29.104677 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 16461
I20260806 14:58:29.115908 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 16642
I20260806 14:58:29.125754 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 16156
2026-08-06T14:58:29Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2115 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 14:58:29 dist-test-slave-68xk krb5kdc[16835](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 14:58:29 dist-test-slave-68xk krb5kdc[16835](info): set up 2 sockets
Aug 06 14:58:29 dist-test-slave-68xk krb5kdc[16835](info): commencing operation
krb5kdc: starting...
W20260806 14:58:30.367352 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 14:58:30 dist-test-slave-68xk krb5kdc[16835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T14:58:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T14:58:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 14:58:30.493368 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:46573
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:37961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:46573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:30.576746 16852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:30.576974 16852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:30.576997 16852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:30.578174 16852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 14:58:30.578219 16852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:30.578233 16852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:30.578246 16852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 14:58:30.578259 16852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 14:58:30.579826 16852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:37961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:46573
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:46573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:30.580087 16852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:30.580346 16852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:30.585405 16861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:30.585387 16860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:30.585634 16852 server_base.cc:1034] running on GCE node
W20260806 14:58:30.585382 16863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:30.586139 16852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:30.586519 16852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:30.587677 16852 hybrid_clock.cc:648] HybridClock initialized: now 1786028310587652 us; error 41 us; skew 500 ppm
Aug 06 14:58:30 dist-test-slave-68xk krb5kdc[16835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028310, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:30.595369 16852 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 14:58:30.595793 16852 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:30.595870 16852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:30.596028 16852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:30.597169 16852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "4767ca4704834281bd4a23e97fc60822"
format_stamp: "Formatted at 2026-08-06 14:58:30 on dist-test-slave-68xk"
server_key: "e926fc986356d3ffea1059144691c37d"
server_key_iv: "07682b4f7b328b8e05a147690c788606"
server_key_version: "encryptionkey@0"
I20260806 14:58:30.597543 16852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "4767ca4704834281bd4a23e97fc60822"
format_stamp: "Formatted at 2026-08-06 14:58:30 on dist-test-slave-68xk"
server_key: "e926fc986356d3ffea1059144691c37d"
server_key_iv: "07682b4f7b328b8e05a147690c788606"
server_key_version: "encryptionkey@0"
I20260806 14:58:30.599063 16852 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 14:58:30.599879 16872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:30.600136 16852 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:30.600267 16852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "4767ca4704834281bd4a23e97fc60822"
format_stamp: "Formatted at 2026-08-06 14:58:30 on dist-test-slave-68xk"
server_key: "e926fc986356d3ffea1059144691c37d"
server_key_iv: "07682b4f7b328b8e05a147690c788606"
server_key_version: "encryptionkey@0"
I20260806 14:58:30.600347 16852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:30.610785 16852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:30.613127 16852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:30.613296 16852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:30.617429 16852 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:46573
I20260806 14:58:30.617452 16938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:46573 every 8 connection(s)
I20260806 14:58:30.617750 16852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 14:58:30.618544 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 16852
I20260806 14:58:30.618650 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 14:58:30.618848 11297 external_mini_cluster.cc:1428] Setting key c30cd6b2497cf9d5c03a733e6cbbe957
I20260806 14:58:30.619109 16939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:30.622684 16939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4767ca4704834281bd4a23e97fc60822: Bootstrap starting.
I20260806 14:58:30.623425 16939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4767ca4704834281bd4a23e97fc60822: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:30.623737 16939 log.cc:824] T 00000000000000000000000000000000 P 4767ca4704834281bd4a23e97fc60822: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:30.624661 16939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4767ca4704834281bd4a23e97fc60822: No bootstrap required, opened a new log
Aug 06 14:58:30 dist-test-slave-68xk krb5kdc[16835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:30.627074 16939 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4767ca4704834281bd4a23e97fc60822 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4767ca4704834281bd4a23e97fc60822" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 46573 } }
I20260806 14:58:30.627260 16939 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4767ca4704834281bd4a23e97fc60822 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:30.627290 16939 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4767ca4704834281bd4a23e97fc60822 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4767ca4704834281bd4a23e97fc60822, State: Initialized, Role: FOLLOWER
I20260806 14:58:30.627444 16939 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4767ca4704834281bd4a23e97fc60822 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4767ca4704834281bd4a23e97fc60822" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 46573 } }
I20260806 14:58:30.627530 16939 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4767ca4704834281bd4a23e97fc60822 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 14:58:30.627581 16939 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4767ca4704834281bd4a23e97fc60822 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 14:58:30.627637 16939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4767ca4704834281bd4a23e97fc60822 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:30.628413 16939 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4767ca4704834281bd4a23e97fc60822 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4767ca4704834281bd4a23e97fc60822" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 46573 } }
I20260806 14:58:30.628536 16939 leader_election.cc:302] T 00000000000000000000000000000000 P 4767ca4704834281bd4a23e97fc60822 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4767ca4704834281bd4a23e97fc60822; no voters:
I20260806 14:58:30.628731 16939 leader_election.cc:288] T 00000000000000000000000000000000 P 4767ca4704834281bd4a23e97fc60822 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 14:58:30.628808 16946 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4767ca4704834281bd4a23e97fc60822 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:30.629001 16939 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4767ca4704834281bd4a23e97fc60822 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 14:58:30.629019 16946 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4767ca4704834281bd4a23e97fc60822 [term 1 LEADER]: Becoming Leader. State: Replica: 4767ca4704834281bd4a23e97fc60822, State: Running, Role: LEADER
I20260806 14:58:30.629175 16946 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4767ca4704834281bd4a23e97fc60822 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4767ca4704834281bd4a23e97fc60822" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 46573 } }
I20260806 14:58:30.629492 16946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4767ca4704834281bd4a23e97fc60822 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4767ca4704834281bd4a23e97fc60822. Latest consensus state: current_term: 1 leader_uuid: "4767ca4704834281bd4a23e97fc60822" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4767ca4704834281bd4a23e97fc60822" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 46573 } } }
I20260806 14:58:30.629567 16946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4767ca4704834281bd4a23e97fc60822 [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:30.629735 16949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4767ca4704834281bd4a23e97fc60822 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4767ca4704834281bd4a23e97fc60822" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4767ca4704834281bd4a23e97fc60822" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 46573 } } }
I20260806 14:58:30.629824 16949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4767ca4704834281bd4a23e97fc60822 [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:30.630057 16957 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 14:58:30.630620 16957 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 14:58:30.631476 16943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:30.619389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34458 (local address 127.11.8.126:46573)
0806 14:58:30.619575 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:30.619590 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:30.619624 (+ 34us) server_negotiation.cc:407] Connection header received
0806 14:58:30.620157 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:30.620166 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:30.620232 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:30.620391 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:30.620967 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:30.622332 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:30.623197 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:30.623837 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:30.623955 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:30.626892 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:30.626896 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:30.626897 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:30.626910 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:30.629182 (+ 2272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:30.629631 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:30.629634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:30.629634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:30.629772 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:30.630016 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:30.630017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:30.630017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:30.630367 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:30.630569 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:30.631135 (+ 566us) server_negotiation.cc:299] Negotiation successful
0806 14:58:30.631288 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":98,"threads_started":1}
I20260806 14:58:30.632382 16957 catalog_manager.cc:1347] Generated new cluster ID: f1198002ad5c47bbba91930fafe57dff
I20260806 14:58:30.632434 16957 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 14:58:30.646391 16957 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 14:58:30.646997 16957 catalog_manager.cc:1504] Loading token signing keys...
I20260806 14:58:30.650307 16957 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4767ca4704834281bd4a23e97fc60822: Generated new TSK 0
I20260806 14:58:30.650527 16957 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
I20260806 14:58:30.685586 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:0
--local_ip_for_outbound_sockets=127.11.8.65
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:46573
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:37961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:30.785629 16977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:30.785859 16977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:30.785902 16977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:30.787156 16977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:30.787223 16977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:30.787269 16977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 14:58:30.788796 16977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:37961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:46573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:30.789062 16977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:30.789304 16977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:30.794495 16987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:30.794538 16986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:30.794765 16990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:30.794989 16977 server_base.cc:1034] running on GCE node
I20260806 14:58:30.795228 16977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:30.795482 16977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:30.796676 16977 hybrid_clock.cc:648] HybridClock initialized: now 1786028310796661 us; error 29 us; skew 500 ppm
Aug 06 14:58:30 dist-test-slave-68xk krb5kdc[16835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028310, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:30.799474 16977 init.cc:377] Logged in from keytab as kudu/127.11.8.65@KRBTEST.COM (short username kudu)
I20260806 14:58:30.799825 16977 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:30.799875 16977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:30.799963 16977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:30.800993 16977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-0/data/instance:
uuid: "c782a4fb49c649faa3974a629d40e590"
format_stamp: "Formatted at 2026-08-06 14:58:30 on dist-test-slave-68xk"
server_key: "493da8571629fae988a8b945110927bb"
server_key_iv: "edddfc9ed6571a5ac6ff4c985c19aecd"
server_key_version: "encryptionkey@0"
I20260806 14:58:30.801405 16977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance:
uuid: "c782a4fb49c649faa3974a629d40e590"
format_stamp: "Formatted at 2026-08-06 14:58:30 on dist-test-slave-68xk"
server_key: "493da8571629fae988a8b945110927bb"
server_key_iv: "edddfc9ed6571a5ac6ff4c985c19aecd"
server_key_version: "encryptionkey@0"
I20260806 14:58:30.802771 16977 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 14:58:30.803665 16999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:30.803882 16977 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:30.804016 16977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "c782a4fb49c649faa3974a629d40e590"
format_stamp: "Formatted at 2026-08-06 14:58:30 on dist-test-slave-68xk"
server_key: "493da8571629fae988a8b945110927bb"
server_key_iv: "edddfc9ed6571a5ac6ff4c985c19aecd"
server_key_version: "encryptionkey@0"
I20260806 14:58:30.804147 16977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:30.814695 16977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:30.817199 16977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:30.817322 16977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:30.817519 16977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:30.817800 16977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:30.817836 16977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:30.817860 16977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:30.817874 16977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:30.824293 16977 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:43027
I20260806 14:58:30.824379 17165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:43027 every 8 connection(s)
I20260806 14:58:30.824693 16977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
Aug 06 14:58:30 dist-test-slave-68xk krb5kdc[16835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028310, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:30.831849 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 16977
I20260806 14:58:30.831957 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance
I20260806 14:58:30.832152 11297 external_mini_cluster.cc:1428] Setting key 6317827d3c03d0c3a282936f3b230d91
I20260806 14:58:30.835448 17169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:30.825471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:46573 (local address 127.11.8.65:43575)
0806 14:58:30.825642 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:30.825685 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:30.826147 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:30.826357 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:30.826361 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:30.826700 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:30.827325 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:30.827331 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:30.828085 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:30.828085 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:30.829241 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:30.829245 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:30.829799 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:30.829800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:30.829913 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:30.830444 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:30.830453 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:30.832051 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:30.833853 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:30.833861 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:30.833863 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:30.834191 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:30.834402 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:30.834403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:30.834403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:30.834606 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:30.834946 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:30.834952 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:30.835083 (+ 131us) client_negotiation.cc:769] Sending connection context
0806 14:58:30.835168 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 14:58:30.835289 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":92,"threads_started":1}
I20260806 14:58:30.835451 16943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:30.825487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:43575 (local address 127.11.8.126:46573)
0806 14:58:30.825557 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:30.825559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:30.825774 (+ 215us) server_negotiation.cc:407] Connection header received
0806 14:58:30.826228 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:30.826229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:30.826240 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:30.826305 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:30.827438 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:30.828000 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:30.829340 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:30.829680 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:30.829778 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:30.832163 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:30.832164 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:30.832164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:30.832172 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:30.833762 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:30.834279 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:30.834280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:30.834281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:30.834333 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:30.834692 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:30.834693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:30.834694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:30.834869 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:30.834949 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:30.835230 (+ 281us) server_negotiation.cc:299] Negotiation successful
0806 14:58:30.835330 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 14:58:30.835985 17166 heartbeater.cc:347] Connected to a master server at 127.11.8.126:46573
I20260806 14:58:30.836081 17166 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:30.836256 17166 heartbeater.cc:511] Master 127.11.8.126:46573 requested a full tablet report, sending...
I20260806 14:58:30.836768 16893 ts_manager.cc:194] Registered new tserver with Master: c782a4fb49c649faa3974a629d40e590 (127.11.8.65:43027)
I20260806 14:58:30.837371 16893 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.65@KRBTEST.COM'} at 127.11.8.65:43575
WARNING: no policy specified for kudu/127.11.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
I20260806 14:58:30.861721 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:0
--local_ip_for_outbound_sockets=127.11.8.66
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:46573
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:37961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:30.945372 17172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:30.945621 17172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:30.945658 17172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:30.946825 17172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:30.946893 17172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:30.946936 17172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 14:58:30.948446 17172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:37961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:46573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:30.948719 17172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:30.948948 17172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:30.953964 17185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:30.953964 17182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:30.953989 17183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:30.954360 17172 server_base.cc:1034] running on GCE node
I20260806 14:58:30.954648 17172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:30.954908 17172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:30.956099 17172 hybrid_clock.cc:648] HybridClock initialized: now 1786028310956079 us; error 30 us; skew 500 ppm
Aug 06 14:58:30 dist-test-slave-68xk krb5kdc[16835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028310, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:30.958761 17172 init.cc:377] Logged in from keytab as kudu/127.11.8.66@KRBTEST.COM (short username kudu)
I20260806 14:58:30.959117 17172 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:30.959164 17172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:30.959316 17172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:30.960286 17172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-1/data/instance:
uuid: "70489ce7b2c24a3e810f32d0be75435c"
format_stamp: "Formatted at 2026-08-06 14:58:30 on dist-test-slave-68xk"
server_key: "689629bed3644399df9320caf7bf1b25"
server_key_iv: "655a84e601e79deb579671d4f62650f4"
server_key_version: "encryptionkey@0"
I20260806 14:58:30.960592 17172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance:
uuid: "70489ce7b2c24a3e810f32d0be75435c"
format_stamp: "Formatted at 2026-08-06 14:58:30 on dist-test-slave-68xk"
server_key: "689629bed3644399df9320caf7bf1b25"
server_key_iv: "655a84e601e79deb579671d4f62650f4"
server_key_version: "encryptionkey@0"
I20260806 14:58:30.961890 17172 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 14:58:30.962757 17192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:30.962981 17172 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:30.963099 17172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "70489ce7b2c24a3e810f32d0be75435c"
format_stamp: "Formatted at 2026-08-06 14:58:30 on dist-test-slave-68xk"
server_key: "689629bed3644399df9320caf7bf1b25"
server_key_iv: "655a84e601e79deb579671d4f62650f4"
server_key_version: "encryptionkey@0"
I20260806 14:58:30.963192 17172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:30.981115 17172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:30.984100 17172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:30.984297 17172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:30.984584 17172 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:30.985009 17172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:30.985085 17172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:30.985121 17172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:30.985152 17172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:30.992039 17172 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:42775
I20260806 14:58:30.992118 17353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:42775 every 8 connection(s)
I20260806 14:58:30.992384 17172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
I20260806 14:58:30.997552 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 17172
I20260806 14:58:30.997700 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance
I20260806 14:58:30.997977 11297 external_mini_cluster.cc:1428] Setting key 42bc0394f94e69b3f5b90ae0dd95310f
Aug 06 14:58:30 dist-test-slave-68xk krb5kdc[16835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028310, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:31.005219 17356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:30.993149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:46573 (local address 127.11.8.66:38219)
0806 14:58:30.993365 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:30.993401 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:30.993894 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:30.994111 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:30.994114 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:30.994493 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:30.995051 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:30.995057 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:30.995754 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:30.995756 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:30.997269 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:30.997272 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:30.997958 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:30.997960 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:30.998168 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:30.998889 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:30.998899 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:31.001123 (+ 2224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:31.003339 (+ 2216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:31.003346 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:31.003348 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:31.003664 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:31.003889 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:31.003890 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:31.003890 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:31.004017 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:31.004350 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:31.004357 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:31.004579 (+ 222us) client_negotiation.cc:769] Sending connection context
0806 14:58:31.004716 (+ 137us) client_negotiation.cc:240] Negotiation successful
0806 14:58:31.004916 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":114,"threads_started":1}
I20260806 14:58:31.005208 16943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:30.993149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:38219 (local address 127.11.8.126:46573)
0806 14:58:30.993234 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:30.993238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:30.993489 (+ 251us) server_negotiation.cc:407] Connection header received
0806 14:58:30.993984 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:30.993986 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:30.993996 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:30.994048 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:30.995154 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:30.995660 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:30.997446 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:30.997816 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:30.997943 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:31.001243 (+ 3300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:31.001245 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:31.001246 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:31.001256 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:31.003212 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:31.003741 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:31.003743 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:31.003743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:31.003815 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:31.004101 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:31.004103 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:31.004103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:31.004238 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:31.004336 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:31.004782 (+ 446us) server_negotiation.cc:299] Negotiation successful
0806 14:58:31.004946 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 14:58:31.005946 17354 heartbeater.cc:347] Connected to a master server at 127.11.8.126:46573
I20260806 14:58:31.006091 17354 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:31.006309 17354 heartbeater.cc:511] Master 127.11.8.126:46573 requested a full tablet report, sending...
I20260806 14:58:31.006958 16893 ts_manager.cc:194] Registered new tserver with Master: 70489ce7b2c24a3e810f32d0be75435c (127.11.8.66:42775)
I20260806 14:58:31.007676 16893 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.66@KRBTEST.COM'} at 127.11.8.66:38219
WARNING: no policy specified for kudu/127.11.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
W20260806 14:58:31.657689 11297 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.11.8.67: real 0.658s user 0.001s sys 0.002s
I20260806 14:58:31.658097 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:0
--local_ip_for_outbound_sockets=127.11.8.67
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:46573
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:37961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:31.742774 17362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:31.742990 17362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:31.743014 17362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:31.744246 17362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:31.744289 17362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:31.744316 17362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 14:58:31.745791 17362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:37961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:46573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:31.745978 17362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:31.746163 17362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 14:58:31.751332 17362 server_base.cc:1034] running on GCE node
W20260806 14:58:31.751302 17372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:31.752278 17370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:31.752673 17367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:31.752954 17362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:31.753222 17362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:31.754390 17362 hybrid_clock.cc:648] HybridClock initialized: now 1786028311754367 us; error 42 us; skew 500 ppm
Aug 06 14:58:31 dist-test-slave-68xk krb5kdc[16835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028311, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:31.757042 17362 init.cc:377] Logged in from keytab as kudu/127.11.8.67@KRBTEST.COM (short username kudu)
I20260806 14:58:31.757442 17362 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:31.757517 17362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:31.757685 17362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:31.758816 17362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-2/data/instance:
uuid: "a96d5bf2f9a34f09b19a0e04623115de"
format_stamp: "Formatted at 2026-08-06 14:58:31 on dist-test-slave-68xk"
server_key: "03326291a9e88784f8cc2ba797bb0eba"
server_key_iv: "d23047cb089297426063bbd0ffeca641"
server_key_version: "encryptionkey@0"
I20260806 14:58:31.759191 17362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance:
uuid: "a96d5bf2f9a34f09b19a0e04623115de"
format_stamp: "Formatted at 2026-08-06 14:58:31 on dist-test-slave-68xk"
server_key: "03326291a9e88784f8cc2ba797bb0eba"
server_key_iv: "d23047cb089297426063bbd0ffeca641"
server_key_version: "encryptionkey@0"
I20260806 14:58:31.760496 17362 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:31.761188 17380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:31.761389 17362 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 14:58:31.761479 17362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "a96d5bf2f9a34f09b19a0e04623115de"
format_stamp: "Formatted at 2026-08-06 14:58:31 on dist-test-slave-68xk"
server_key: "03326291a9e88784f8cc2ba797bb0eba"
server_key_iv: "d23047cb089297426063bbd0ffeca641"
server_key_version: "encryptionkey@0"
I20260806 14:58:31.761566 17362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:31.778358 17362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:31.780345 17362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:31.780515 17362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:31.780768 17362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:31.781101 17362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:31.781157 17362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:31.781199 17362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:31.781230 17362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:31.787552 17362 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:37067
I20260806 14:58:31.787587 17529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:37067 every 8 connection(s)
I20260806 14:58:31.787871 17362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
Aug 06 14:58:31 dist-test-slave-68xk krb5kdc[16835](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028311, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:31.794381 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 17362
I20260806 14:58:31.794516 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance
I20260806 14:58:31.794749 11297 external_mini_cluster.cc:1428] Setting key 291848bb83c2adaed2e6018dbd912490
I20260806 14:58:31.798130 17533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:31.788798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:39189 (local address 127.11.8.126:46573)
0806 14:58:31.788951 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:31.788953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:31.789002 (+ 49us) server_negotiation.cc:407] Connection header received
0806 14:58:31.789551 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:31.789553 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:31.789573 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:31.789677 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:31.790702 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:31.791264 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:31.792414 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:31.792656 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:31.792719 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:31.795086 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:31.795087 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:31.795088 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:31.795096 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:31.796669 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:31.797106 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:31.797107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:31.797108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:31.797175 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:31.797436 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:31.797437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:31.797437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:31.797562 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:31.797642 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:31.797919 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 14:58:31.798010 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":95,"threads_started":1}
I20260806 14:58:31.798295 17532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:31.788715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:46573 (local address 127.11.8.67:39189)
0806 14:58:31.788884 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:31.788929 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:31.789440 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:31.789775 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:31.789779 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:31.790048 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:31.790607 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:31.790613 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:31.791369 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:31.791370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:31.792322 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:31.792324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:31.792738 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:31.792739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:31.792842 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:31.793313 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:31.793321 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:31.794975 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:31.796759 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:31.796767 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:31.796769 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:31.797005 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:31.797248 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:31.797248 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:31.797249 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:31.797364 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:31.797633 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:31.797640 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:31.797784 (+ 144us) client_negotiation.cc:769] Sending connection context
0806 14:58:31.797897 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 14:58:31.798057 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":91,"threads_started":1}
I20260806 14:58:31.798799 17530 heartbeater.cc:347] Connected to a master server at 127.11.8.126:46573
I20260806 14:58:31.798887 17530 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:31.799063 17530 heartbeater.cc:511] Master 127.11.8.126:46573 requested a full tablet report, sending...
I20260806 14:58:31.799490 16893 ts_manager.cc:194] Registered new tserver with Master: a96d5bf2f9a34f09b19a0e04623115de (127.11.8.67:37067)
I20260806 14:58:31.799849 16893 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.67@KRBTEST.COM'} at 127.11.8.67:39189
I20260806 14:58:31.806803 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 14:58:31.814771 17533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:31.808277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47196 (local address 127.11.8.126:46573)
0806 14:58:31.808353 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:31.808356 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:31.808486 (+ 130us) server_negotiation.cc:407] Connection header received
0806 14:58:31.808525 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:31.808525 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:31.808536 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:31.808598 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:31.809166 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:31.809707 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:31.810490 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:31.810717 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:31.810779 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:31.811960 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:31.811961 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:31.811962 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:31.811969 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:31.813638 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:31.813938 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:31.813938 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:31.813939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:31.813993 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:31.814141 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:31.814141 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:31.814142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:31.814281 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:31.814361 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:31.814545 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 14:58:31.814651 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 14:58:31.816708 16893 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 14:58:31.818306 16893 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 14:58:31.825366 17547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:31.821185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:42775 (local address 127.0.0.1:36548)
0806 14:58:31.821438 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:31.821479 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:31.821565 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:31.821835 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:31.821839 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:31.821841 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:31.821915 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:31.821918 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:31.823087 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:31.823088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:31.824124 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:31.824127 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:31.825064 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:31.825065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:31.825146 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:31.825160 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 14:58:31.825234 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 14:58:31.825290 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":152,"threads_started":1}
I20260806 14:58:31.825385 17549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:31.821316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36548 (local address 127.11.8.66:42775)
0806 14:58:31.821459 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:31.821464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:31.821556 (+ 92us) server_negotiation.cc:407] Connection header received
0806 14:58:31.821663 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:31.821666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:31.821683 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:31.821829 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:31.822017 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:31.822999 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:31.824226 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:31.824964 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:31.825063 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:31.825129 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:31.825258 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 14:58:31.825310 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":53,"threads_started":1}
I20260806 14:58:31.825961 17553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:31.821471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:43027 (local address 127.0.0.1:42786)
0806 14:58:31.822182 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:31.822192 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:31.822254 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:31.822481 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:31.822483 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:31.822485 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:31.822538 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:31.822539 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:31.823725 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:31.823726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:31.824776 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:31.824780 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:31.825700 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:31.825702 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:31.825770 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:31.825775 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 14:58:31.825837 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 14:58:31.825879 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"thread_start_us":67,"threads_started":1}
I20260806 14:58:31.826001 17554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:31.821681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42786 (local address 127.11.8.65:43027)
0806 14:58:31.821829 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:31.821834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:31.822311 (+ 477us) server_negotiation.cc:407] Connection header received
0806 14:58:31.822349 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:31.822351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:31.822367 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:31.822470 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:31.822602 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:31.823630 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:31.824875 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:31.825540 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:31.825610 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:31.825661 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:31.825871 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 14:58:31.825924 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":81,"threads_started":1}
I20260806 14:58:31.826329 17260 tablet_service.cc:1467] Processing CreateTablet for tablet b3d5bc25778e486ba41013db15112e32 (DEFAULT_TABLE table=test-table [id=014f3f0523f54c4bb68888a926902938]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:31.826730 17260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3d5bc25778e486ba41013db15112e32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:31.826792 17068 tablet_service.cc:1467] Processing CreateTablet for tablet b3d5bc25778e486ba41013db15112e32 (DEFAULT_TABLE table=test-table [id=014f3f0523f54c4bb68888a926902938]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:31.827126 17068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3d5bc25778e486ba41013db15112e32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:31.827149 17548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:31.821221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:37067 (local address 127.0.0.1:52620)
0806 14:58:31.821512 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:31.821521 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:31.821565 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:31.822073 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:31.822074 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:31.822075 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:31.822130 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:31.822131 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:31.823462 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:31.823463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:31.824458 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:31.824461 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:31.826902 (+ 2441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:31.826903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:31.826977 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:31.826981 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 14:58:31.827041 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 14:58:31.827082 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":94,"threads_started":1}
I20260806 14:58:31.827174 17550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:31.821346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52620 (local address 127.11.8.67:37067)
0806 14:58:31.821916 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:31.821921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:31.821931 (+ 10us) server_negotiation.cc:407] Connection header received
0806 14:58:31.821968 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:31.821970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:31.821987 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:31.822075 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:31.822213 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:31.823344 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:31.826050 (+ 2706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:31.826779 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:31.826895 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:31.826953 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:31.827056 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 14:58:31.827091 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":105,"threads_started":1}
I20260806 14:58:31.827796 17447 tablet_service.cc:1467] Processing CreateTablet for tablet b3d5bc25778e486ba41013db15112e32 (DEFAULT_TABLE table=test-table [id=014f3f0523f54c4bb68888a926902938]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:31.828083 17447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3d5bc25778e486ba41013db15112e32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:31.830629 17556 tablet_bootstrap.cc:492] T b3d5bc25778e486ba41013db15112e32 P 70489ce7b2c24a3e810f32d0be75435c: Bootstrap starting.
I20260806 14:58:31.830689 17555 tablet_bootstrap.cc:492] T b3d5bc25778e486ba41013db15112e32 P c782a4fb49c649faa3974a629d40e590: Bootstrap starting.
I20260806 14:58:31.830994 17557 tablet_bootstrap.cc:492] T b3d5bc25778e486ba41013db15112e32 P a96d5bf2f9a34f09b19a0e04623115de: Bootstrap starting.
I20260806 14:58:31.831439 17555 tablet_bootstrap.cc:654] T b3d5bc25778e486ba41013db15112e32 P c782a4fb49c649faa3974a629d40e590: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:31.831522 17556 tablet_bootstrap.cc:654] T b3d5bc25778e486ba41013db15112e32 P 70489ce7b2c24a3e810f32d0be75435c: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:31.831683 17557 tablet_bootstrap.cc:654] T b3d5bc25778e486ba41013db15112e32 P a96d5bf2f9a34f09b19a0e04623115de: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:31.831776 17555 log.cc:824] T b3d5bc25778e486ba41013db15112e32 P c782a4fb49c649faa3974a629d40e590: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:31.831854 17556 log.cc:824] T b3d5bc25778e486ba41013db15112e32 P 70489ce7b2c24a3e810f32d0be75435c: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:31.832397 17557 log.cc:824] T b3d5bc25778e486ba41013db15112e32 P a96d5bf2f9a34f09b19a0e04623115de: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:31.832681 17555 tablet_bootstrap.cc:492] T b3d5bc25778e486ba41013db15112e32 P c782a4fb49c649faa3974a629d40e590: No bootstrap required, opened a new log
I20260806 14:58:31.832769 17555 ts_tablet_manager.cc:1397] T b3d5bc25778e486ba41013db15112e32 P c782a4fb49c649faa3974a629d40e590: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 14:58:31.832863 17556 tablet_bootstrap.cc:492] T b3d5bc25778e486ba41013db15112e32 P 70489ce7b2c24a3e810f32d0be75435c: No bootstrap required, opened a new log
I20260806 14:58:31.832968 17556 ts_tablet_manager.cc:1397] T b3d5bc25778e486ba41013db15112e32 P 70489ce7b2c24a3e810f32d0be75435c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 14:58:31.833209 17557 tablet_bootstrap.cc:492] T b3d5bc25778e486ba41013db15112e32 P a96d5bf2f9a34f09b19a0e04623115de: No bootstrap required, opened a new log
I20260806 14:58:31.833267 17557 ts_tablet_manager.cc:1397] T b3d5bc25778e486ba41013db15112e32 P a96d5bf2f9a34f09b19a0e04623115de: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 14:58:31.834663 17557 raft_consensus.cc:348] T b3d5bc25778e486ba41013db15112e32 P a96d5bf2f9a34f09b19a0e04623115de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70489ce7b2c24a3e810f32d0be75435c" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 42775 } } peers { permanent_uuid: "c782a4fb49c649faa3974a629d40e590" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43027 } } peers { permanent_uuid: "a96d5bf2f9a34f09b19a0e04623115de" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37067 } }
I20260806 14:58:31.834802 17557 raft_consensus.cc:374] T b3d5bc25778e486ba41013db15112e32 P a96d5bf2f9a34f09b19a0e04623115de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:31.834833 17557 raft_consensus.cc:729] T b3d5bc25778e486ba41013db15112e32 P a96d5bf2f9a34f09b19a0e04623115de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a96d5bf2f9a34f09b19a0e04623115de, State: Initialized, Role: FOLLOWER
I20260806 14:58:31.834949 17557 consensus_queue.cc:260] T b3d5bc25778e486ba41013db15112e32 P a96d5bf2f9a34f09b19a0e04623115de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70489ce7b2c24a3e810f32d0be75435c" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 42775 } } peers { permanent_uuid: "c782a4fb49c649faa3974a629d40e590" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43027 } } peers { permanent_uuid: "a96d5bf2f9a34f09b19a0e04623115de" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37067 } }
I20260806 14:58:31.835237 17557 ts_tablet_manager.cc:1428] T b3d5bc25778e486ba41013db15112e32 P a96d5bf2f9a34f09b19a0e04623115de: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 14:58:31.835237 17530 heartbeater.cc:503] Master 127.11.8.126:46573 was elected leader, sending a full tablet report...
I20260806 14:58:31.835520 17555 raft_consensus.cc:348] T b3d5bc25778e486ba41013db15112e32 P c782a4fb49c649faa3974a629d40e590 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70489ce7b2c24a3e810f32d0be75435c" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 42775 } } peers { permanent_uuid: "c782a4fb49c649faa3974a629d40e590" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43027 } } peers { permanent_uuid: "a96d5bf2f9a34f09b19a0e04623115de" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37067 } }
I20260806 14:58:31.835680 17555 raft_consensus.cc:374] T b3d5bc25778e486ba41013db15112e32 P c782a4fb49c649faa3974a629d40e590 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:31.835724 17555 raft_consensus.cc:729] T b3d5bc25778e486ba41013db15112e32 P c782a4fb49c649faa3974a629d40e590 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c782a4fb49c649faa3974a629d40e590, State: Initialized, Role: FOLLOWER
I20260806 14:58:31.835858 17555 consensus_queue.cc:260] T b3d5bc25778e486ba41013db15112e32 P c782a4fb49c649faa3974a629d40e590 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70489ce7b2c24a3e810f32d0be75435c" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 42775 } } peers { permanent_uuid: "c782a4fb49c649faa3974a629d40e590" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43027 } } peers { permanent_uuid: "a96d5bf2f9a34f09b19a0e04623115de" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37067 } }
I20260806 14:58:31.836136 17555 ts_tablet_manager.cc:1428] T b3d5bc25778e486ba41013db15112e32 P c782a4fb49c649faa3974a629d40e590: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 14:58:31.836193 17166 heartbeater.cc:503] Master 127.11.8.126:46573 was elected leader, sending a full tablet report...
I20260806 14:58:31.836474 17556 raft_consensus.cc:348] T b3d5bc25778e486ba41013db15112e32 P 70489ce7b2c24a3e810f32d0be75435c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70489ce7b2c24a3e810f32d0be75435c" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 42775 } } peers { permanent_uuid: "c782a4fb49c649faa3974a629d40e590" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43027 } } peers { permanent_uuid: "a96d5bf2f9a34f09b19a0e04623115de" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37067 } }
I20260806 14:58:31.836630 17556 raft_consensus.cc:374] T b3d5bc25778e486ba41013db15112e32 P 70489ce7b2c24a3e810f32d0be75435c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:31.836663 17556 raft_consensus.cc:729] T b3d5bc25778e486ba41013db15112e32 P 70489ce7b2c24a3e810f32d0be75435c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70489ce7b2c24a3e810f32d0be75435c, State: Initialized, Role: FOLLOWER
I20260806 14:58:31.836776 17556 consensus_queue.cc:260] T b3d5bc25778e486ba41013db15112e32 P 70489ce7b2c24a3e810f32d0be75435c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70489ce7b2c24a3e810f32d0be75435c" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 42775 } } peers { permanent_uuid: "c782a4fb49c649faa3974a629d40e590" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43027 } } peers { permanent_uuid: "a96d5bf2f9a34f09b19a0e04623115de" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37067 } }
I20260806 14:58:31.837044 17556 ts_tablet_manager.cc:1428] T b3d5bc25778e486ba41013db15112e32 P 70489ce7b2c24a3e810f32d0be75435c: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260806 14:58:31.837117 17354 heartbeater.cc:503] Master 127.11.8.126:46573 was elected leader, sending a full tablet report...
I20260806 14:58:31.841233 17562 raft_consensus.cc:482] T b3d5bc25778e486ba41013db15112e32 P c782a4fb49c649faa3974a629d40e590 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 14:58:31.841318 17562 raft_consensus.cc:504] T b3d5bc25778e486ba41013db15112e32 P c782a4fb49c649faa3974a629d40e590 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70489ce7b2c24a3e810f32d0be75435c" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 42775 } } peers { permanent_uuid: "c782a4fb49c649faa3974a629d40e590" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43027 } } peers { permanent_uuid: "a96d5bf2f9a34f09b19a0e04623115de" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37067 } }
I20260806 14:58:31.841606 17562 leader_election.cc:288] T b3d5bc25778e486ba41013db15112e32 P c782a4fb49c649faa3974a629d40e590 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70489ce7b2c24a3e810f32d0be75435c (127.11.8.66:42775), a96d5bf2f9a34f09b19a0e04623115de (127.11.8.67:37067)
I20260806 14:58:31.845060 17549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:31.841878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:34155 (local address 127.11.8.66:42775)
0806 14:58:31.841929 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:31.841931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:31.842156 (+ 225us) server_negotiation.cc:407] Connection header received
0806 14:58:31.842185 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:31.842186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:31.842199 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:31.842291 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:31.842510 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:31.842994 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:31.844332 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:31.844686 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:31.844771 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:31.844781 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:31.844948 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 14:58:31.844986 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 14:58:31.845080 17565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:31.841774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:42775 (local address 127.11.8.65:34155)
0806 14:58:31.842041 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:31.842055 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:31.842121 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:31.842311 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:31.842313 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:31.842316 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:31.842435 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:31.842438 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:31.843070 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:31.843071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:31.844236 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:31.844239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:31.844780 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:31.844781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:31.844857 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:31.844860 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 14:58:31.844938 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 14:58:31.844978 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":149,"threads_started":1}
I20260806 14:58:31.845384 17294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3d5bc25778e486ba41013db15112e32" candidate_uuid: "c782a4fb49c649faa3974a629d40e590" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70489ce7b2c24a3e810f32d0be75435c" is_pre_election: true
I20260806 14:58:31.845535 17294 raft_consensus.cc:2454] T b3d5bc25778e486ba41013db15112e32 P 70489ce7b2c24a3e810f32d0be75435c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c782a4fb49c649faa3974a629d40e590 in term 0.
I20260806 14:58:31.845778 17566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:31.841792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:37067 (local address 127.11.8.65:47983)
0806 14:58:31.842223 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:31.842233 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:31.842305 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:31.843080 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:31.843081 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:31.843083 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:31.843138 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:31.843139 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:31.844016 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:31.844017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:31.844751 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:31.844753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:31.845488 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:31.845489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:31.845570 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:31.845573 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 14:58:31.845678 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 14:58:31.845722 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":126,"threads_started":1}
I20260806 14:58:31.845785 17005 leader_election.cc:302] T b3d5bc25778e486ba41013db15112e32 P c782a4fb49c649faa3974a629d40e590 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70489ce7b2c24a3e810f32d0be75435c, c782a4fb49c649faa3974a629d40e590; no voters:
I20260806 14:58:31.845937 17550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:31.841909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:47983 (local address 127.11.8.67:37067)
0806 14:58:31.841966 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:31.841968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:31.842278 (+ 310us) server_negotiation.cc:407] Connection header received
0806 14:58:31.842431 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:31.842433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:31.842447 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:31.842502 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:31.843212 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:31.843928 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:31.844882 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:31.845395 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:31.845487 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:31.845496 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:31.845824 (+ 328us) server_negotiation.cc:299] Negotiation successful
0806 14:58:31.845875 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 14:58:31.845968 17562 raft_consensus.cc:2781] T b3d5bc25778e486ba41013db15112e32 P c782a4fb49c649faa3974a629d40e590 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 14:58:31.846027 17562 raft_consensus.cc:482] T b3d5bc25778e486ba41013db15112e32 P c782a4fb49c649faa3974a629d40e590 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 14:58:31.846055 17562 raft_consensus.cc:3037] T b3d5bc25778e486ba41013db15112e32 P c782a4fb49c649faa3974a629d40e590 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:31.846184 17472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3d5bc25778e486ba41013db15112e32" candidate_uuid: "c782a4fb49c649faa3974a629d40e590" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a96d5bf2f9a34f09b19a0e04623115de" is_pre_election: true
I20260806 14:58:31.846302 17472 raft_consensus.cc:2454] T b3d5bc25778e486ba41013db15112e32 P a96d5bf2f9a34f09b19a0e04623115de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c782a4fb49c649faa3974a629d40e590 in term 0.
I20260806 14:58:31.846877 17562 raft_consensus.cc:504] T b3d5bc25778e486ba41013db15112e32 P c782a4fb49c649faa3974a629d40e590 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70489ce7b2c24a3e810f32d0be75435c" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 42775 } } peers { permanent_uuid: "c782a4fb49c649faa3974a629d40e590" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43027 } } peers { permanent_uuid: "a96d5bf2f9a34f09b19a0e04623115de" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37067 } }
I20260806 14:58:31.847025 17562 leader_election.cc:288] T b3d5bc25778e486ba41013db15112e32 P c782a4fb49c649faa3974a629d40e590 [CANDIDATE]: Term 1 election: Requested vote from peers 70489ce7b2c24a3e810f32d0be75435c (127.11.8.66:42775), a96d5bf2f9a34f09b19a0e04623115de (127.11.8.67:37067)
I20260806 14:58:31.847185 17472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3d5bc25778e486ba41013db15112e32" candidate_uuid: "c782a4fb49c649faa3974a629d40e590" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a96d5bf2f9a34f09b19a0e04623115de"
I20260806 14:58:31.847198 17294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3d5bc25778e486ba41013db15112e32" candidate_uuid: "c782a4fb49c649faa3974a629d40e590" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70489ce7b2c24a3e810f32d0be75435c"
I20260806 14:58:31.847265 17472 raft_consensus.cc:3037] T b3d5bc25778e486ba41013db15112e32 P a96d5bf2f9a34f09b19a0e04623115de [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:31.847291 17294 raft_consensus.cc:3037] T b3d5bc25778e486ba41013db15112e32 P 70489ce7b2c24a3e810f32d0be75435c [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:31.847949 17472 raft_consensus.cc:2454] T b3d5bc25778e486ba41013db15112e32 P a96d5bf2f9a34f09b19a0e04623115de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c782a4fb49c649faa3974a629d40e590 in term 1.
I20260806 14:58:31.848120 17294 raft_consensus.cc:2454] T b3d5bc25778e486ba41013db15112e32 P 70489ce7b2c24a3e810f32d0be75435c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c782a4fb49c649faa3974a629d40e590 in term 1.
I20260806 14:58:31.848151 17002 leader_election.cc:302] T b3d5bc25778e486ba41013db15112e32 P c782a4fb49c649faa3974a629d40e590 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a96d5bf2f9a34f09b19a0e04623115de, c782a4fb49c649faa3974a629d40e590; no voters:
I20260806 14:58:31.848249 17562 raft_consensus.cc:2781] T b3d5bc25778e486ba41013db15112e32 P c782a4fb49c649faa3974a629d40e590 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:31.848400 17562 raft_consensus.cc:686] T b3d5bc25778e486ba41013db15112e32 P c782a4fb49c649faa3974a629d40e590 [term 1 LEADER]: Becoming Leader. State: Replica: c782a4fb49c649faa3974a629d40e590, State: Running, Role: LEADER
I20260806 14:58:31.848503 17562 consensus_queue.cc:237] T b3d5bc25778e486ba41013db15112e32 P c782a4fb49c649faa3974a629d40e590 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70489ce7b2c24a3e810f32d0be75435c" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 42775 } } peers { permanent_uuid: "c782a4fb49c649faa3974a629d40e590" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43027 } } peers { permanent_uuid: "a96d5bf2f9a34f09b19a0e04623115de" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37067 } }
I20260806 14:58:31.849323 16893 catalog_manager.cc:5495] T b3d5bc25778e486ba41013db15112e32 P c782a4fb49c649faa3974a629d40e590 reported cstate change: term changed from 0 to 1, leader changed from <none> to c782a4fb49c649faa3974a629d40e590 (127.11.8.65). New cstate: current_term: 1 leader_uuid: "c782a4fb49c649faa3974a629d40e590" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70489ce7b2c24a3e810f32d0be75435c" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 42775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c782a4fb49c649faa3974a629d40e590" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a96d5bf2f9a34f09b19a0e04623115de" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37067 } health_report { overall_health: UNKNOWN } } }
I20260806 14:58:31.861096 17554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:31.858790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42802 (local address 127.11.8.65:43027)
0806 14:58:31.858850 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:31.858852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:31.858889 (+ 37us) server_negotiation.cc:407] Connection header received
0806 14:58:31.858919 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:31.858919 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:31.858936 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:31.858990 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 14:58:31.859096 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:31.859548 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:31.860260 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:31.860441 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:31.860499 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:31.860668 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 14:58:31.860765 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 14:58:31.860828 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:31.860860 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 14:58:31.860878 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 14:58:31.863034 17472 raft_consensus.cc:1261] T b3d5bc25778e486ba41013db15112e32 P a96d5bf2f9a34f09b19a0e04623115de [term 1 FOLLOWER]: Refusing update from remote peer c782a4fb49c649faa3974a629d40e590: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:31.863082 17294 raft_consensus.cc:1261] T b3d5bc25778e486ba41013db15112e32 P 70489ce7b2c24a3e810f32d0be75435c [term 1 FOLLOWER]: Refusing update from remote peer c782a4fb49c649faa3974a629d40e590: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:31.863402 17562 consensus_queue.cc:1035] T b3d5bc25778e486ba41013db15112e32 P c782a4fb49c649faa3974a629d40e590 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a96d5bf2f9a34f09b19a0e04623115de" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:31.863533 17567 consensus_queue.cc:1035] T b3d5bc25778e486ba41013db15112e32 P c782a4fb49c649faa3974a629d40e590 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70489ce7b2c24a3e810f32d0be75435c" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 42775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:31.866294 17575 mvcc.cc:205] Tried to move back new op lower bound from 7315571965387960320 to 7315571965332271104. Current Snapshot: MvccSnapshot[applied={T|T < 7315571965387960320}]
I20260806 14:58:31.866622 17572 mvcc.cc:205] Tried to move back new op lower bound from 7315571965387960320 to 7315571965332271104. Current Snapshot: MvccSnapshot[applied={T|T < 7315571965387960320}]
I20260806 14:58:31.866855 17574 mvcc.cc:205] Tried to move back new op lower bound from 7315571965387960320 to 7315571965332271104. Current Snapshot: MvccSnapshot[applied={T|T < 7315571965387960320}]
I20260806 14:58:31.869117 16892 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:58:31.869217 16892 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:58:31.869976 16892 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4767ca4704834281bd4a23e97fc60822: Sending DeleteTablet for 3 replicas of tablet b3d5bc25778e486ba41013db15112e32
I20260806 14:58:31.870301 17260 tablet_service.cc:1514] Processing DeleteTablet for tablet b3d5bc25778e486ba41013db15112e32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:31 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:36548
I20260806 14:58:31.870342 17068 tablet_service.cc:1514] Processing DeleteTablet for tablet b3d5bc25778e486ba41013db15112e32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:31 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:42786
I20260806 14:58:31.870330 17447 tablet_service.cc:1514] Processing DeleteTablet for tablet b3d5bc25778e486ba41013db15112e32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:31 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:52620
I20260806 14:58:31.870573 17588 tablet_replica.cc:330] T b3d5bc25778e486ba41013db15112e32 P a96d5bf2f9a34f09b19a0e04623115de: stopping tablet replica
I20260806 14:58:31.870687 17588 raft_consensus.cc:2229] T b3d5bc25778e486ba41013db15112e32 P a96d5bf2f9a34f09b19a0e04623115de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:58:31.870715 17589 tablet_replica.cc:330] T b3d5bc25778e486ba41013db15112e32 P c782a4fb49c649faa3974a629d40e590: stopping tablet replica
I20260806 14:58:31.870800 17588 raft_consensus.cc:2258] T b3d5bc25778e486ba41013db15112e32 P a96d5bf2f9a34f09b19a0e04623115de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:31.870959 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 16977
I20260806 14:58:31.881315 17588 ts_tablet_manager.cc:1905] T b3d5bc25778e486ba41013db15112e32 P a96d5bf2f9a34f09b19a0e04623115de: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:31.882869 17587 tablet_replica.cc:330] T b3d5bc25778e486ba41013db15112e32 P 70489ce7b2c24a3e810f32d0be75435c: stopping tablet replica
I20260806 14:58:31.883023 17587 raft_consensus.cc:2229] T b3d5bc25778e486ba41013db15112e32 P 70489ce7b2c24a3e810f32d0be75435c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:58:31.883154 17587 raft_consensus.cc:2258] T b3d5bc25778e486ba41013db15112e32 P 70489ce7b2c24a3e810f32d0be75435c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:31.883476 17588 ts_tablet_manager.cc:1918] T b3d5bc25778e486ba41013db15112e32 P a96d5bf2f9a34f09b19a0e04623115de: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 14:58:31.883567 17588 log.cc:1196] T b3d5bc25778e486ba41013db15112e32 P a96d5bf2f9a34f09b19a0e04623115de: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786028287744255-11297-0/minicluster-data/ts-2/wal/wals/b3d5bc25778e486ba41013db15112e32
I20260806 14:58:31.883876 17588 ts_tablet_manager.cc:1939] T b3d5bc25778e486ba41013db15112e32 P a96d5bf2f9a34f09b19a0e04623115de: Deleting consensus metadata
I20260806 14:58:31.884347 16873 catalog_manager.cc:4841] TS a96d5bf2f9a34f09b19a0e04623115de (127.11.8.67:37067): tablet b3d5bc25778e486ba41013db15112e32 (table test-table [id=014f3f0523f54c4bb68888a926902938]) successfully deleted
W20260806 14:58:31.887394 16876 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.65:43027 (error 108)
I20260806 14:58:31.887671 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 17172
I20260806 14:58:31.889391 17548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:31.889251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:43027 (local address 127.0.0.1:42816)
0806 14:58:31.889316 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:31.889362 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.65:43027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 14:58:31.889593 16876 catalog_manager.cc:4584] TS c782a4fb49c649faa3974a629d40e590 (127.11.8.65:43027): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b3d5bc25778e486ba41013db15112e32: Network error: Client connection negotiation failed: client connection to 127.11.8.65:43027: connect: Connection refused (error 111)
I20260806 14:58:31.898155 17548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:31.898024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:42775 (local address 127.0.0.1:36562)
0806 14:58:31.898089 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:31.898126 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.66:42775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 14:58:31.898165 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 17362
I20260806 14:58:31.903856 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 16852
2026-08-06T14:58:31Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2781 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T14:58:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T14:58:31Z Disabled control of system clock
I20260806 14:58:31.937908 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:46685
--webserver_interface=127.11.8.126
--webserver_port=0
--builtin_ntp_servers=127.11.8.84:38953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:46685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 14:58:32.020268 17595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:32.020478 17595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:32.020500 17595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:32.021554 17595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 14:58:32.021590 17595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:32.021603 17595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:32.021615 17595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 14:58:32.021628 17595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 14:58:32.022989 17595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:38953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:46685
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:46685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:32.023191 17595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:32.023432 17595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 14:58:32.028352 17595 server_base.cc:1034] running on GCE node
W20260806 14:58:32.028321 17605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:32.028314 17603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:32.028326 17608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:32.028785 17595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:32.029021 17595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:32.030196 17595 hybrid_clock.cc:648] HybridClock initialized: now 1786028312030187 us; error 29 us; skew 500 ppm
I20260806 14:58:32.031509 17595 webserver.cc:460] Webserver started at http://127.11.8.126:33863/ using document root <none> and password file <none>
I20260806 14:58:32.031793 17595 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:32.031843 17595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:32.032027 17595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:32.033095 17595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "89b73c91ce6a4efdbbafcf8f943f07d6"
format_stamp: "Formatted at 2026-08-06 14:58:32 on dist-test-slave-68xk"
server_key: "aa16f5cc1a9134db88000d29f9fb14db"
server_key_iv: "5d88484206372a676c64fa5473953291"
server_key_version: "encryptionkey@0"
I20260806 14:58:32.033465 17595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "89b73c91ce6a4efdbbafcf8f943f07d6"
format_stamp: "Formatted at 2026-08-06 14:58:32 on dist-test-slave-68xk"
server_key: "aa16f5cc1a9134db88000d29f9fb14db"
server_key_iv: "5d88484206372a676c64fa5473953291"
server_key_version: "encryptionkey@0"
I20260806 14:58:32.034893 17595 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 14:58:32.035722 17614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:32.036036 17595 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:32.036132 17595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "89b73c91ce6a4efdbbafcf8f943f07d6"
format_stamp: "Formatted at 2026-08-06 14:58:32 on dist-test-slave-68xk"
server_key: "aa16f5cc1a9134db88000d29f9fb14db"
server_key_iv: "5d88484206372a676c64fa5473953291"
server_key_version: "encryptionkey@0"
I20260806 14:58:32.036206 17595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:32.039592 17595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:32.039855 17595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:32.040005 17595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:32.043987 17595 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:46685
I20260806 14:58:32.044020 17687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:46685 every 8 connection(s)
I20260806 14:58:32.044340 17595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 14:58:32.045038 17688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:32.047940 17688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89b73c91ce6a4efdbbafcf8f943f07d6: Bootstrap starting.
I20260806 14:58:32.048511 17688 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89b73c91ce6a4efdbbafcf8f943f07d6: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:32.048796 17688 log.cc:824] T 00000000000000000000000000000000 P 89b73c91ce6a4efdbbafcf8f943f07d6: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:32.049464 17688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89b73c91ce6a4efdbbafcf8f943f07d6: No bootstrap required, opened a new log
I20260806 14:58:32.051126 17688 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89b73c91ce6a4efdbbafcf8f943f07d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b73c91ce6a4efdbbafcf8f943f07d6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 46685 } }
I20260806 14:58:32.051265 17688 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89b73c91ce6a4efdbbafcf8f943f07d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:32.051316 17688 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89b73c91ce6a4efdbbafcf8f943f07d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89b73c91ce6a4efdbbafcf8f943f07d6, State: Initialized, Role: FOLLOWER
I20260806 14:58:32.051455 17688 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89b73c91ce6a4efdbbafcf8f943f07d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b73c91ce6a4efdbbafcf8f943f07d6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 46685 } }
I20260806 14:58:32.051514 17688 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89b73c91ce6a4efdbbafcf8f943f07d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 14:58:32.051553 17688 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89b73c91ce6a4efdbbafcf8f943f07d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 14:58:32.051600 17688 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89b73c91ce6a4efdbbafcf8f943f07d6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:32.052119 17688 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89b73c91ce6a4efdbbafcf8f943f07d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b73c91ce6a4efdbbafcf8f943f07d6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 46685 } }
I20260806 14:58:32.052237 17688 leader_election.cc:302] T 00000000000000000000000000000000 P 89b73c91ce6a4efdbbafcf8f943f07d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89b73c91ce6a4efdbbafcf8f943f07d6; no voters:
I20260806 14:58:32.052435 17688 leader_election.cc:288] T 00000000000000000000000000000000 P 89b73c91ce6a4efdbbafcf8f943f07d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 14:58:32.052464 17691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89b73c91ce6a4efdbbafcf8f943f07d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:32.052449 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 17595
I20260806 14:58:32.052551 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 14:58:32.052588 17691 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89b73c91ce6a4efdbbafcf8f943f07d6 [term 1 LEADER]: Becoming Leader. State: Replica: 89b73c91ce6a4efdbbafcf8f943f07d6, State: Running, Role: LEADER
I20260806 14:58:32.052703 17688 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89b73c91ce6a4efdbbafcf8f943f07d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 14:58:32.052680 17691 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89b73c91ce6a4efdbbafcf8f943f07d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b73c91ce6a4efdbbafcf8f943f07d6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 46685 } }
I20260806 14:58:32.052767 11297 external_mini_cluster.cc:1428] Setting key 803cdfe630bb1ef1a22a2703d3d13ef1
I20260806 14:58:32.053251 17692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89b73c91ce6a4efdbbafcf8f943f07d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89b73c91ce6a4efdbbafcf8f943f07d6. Latest consensus state: current_term: 1 leader_uuid: "89b73c91ce6a4efdbbafcf8f943f07d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b73c91ce6a4efdbbafcf8f943f07d6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 46685 } } }
I20260806 14:58:32.053346 17692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89b73c91ce6a4efdbbafcf8f943f07d6 [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:32.053604 17693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89b73c91ce6a4efdbbafcf8f943f07d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89b73c91ce6a4efdbbafcf8f943f07d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89b73c91ce6a4efdbbafcf8f943f07d6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 46685 } } }
I20260806 14:58:32.053673 17693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89b73c91ce6a4efdbbafcf8f943f07d6 [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:32.054054 17705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 14:58:32.055493 17705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 14:58:32.055783 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:32.053501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34876 (local address 127.11.8.126:46685)
0806 14:58:32.054234 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:32.054243 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:32.054277 (+ 34us) server_negotiation.cc:407] Connection header received
0806 14:58:32.054397 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:32.054403 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:32.054511 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:32.054648 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:32.055298 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:32.055302 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:32.055303 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 14:58:32.055308 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:32.055356 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 14:58:32.055371 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:32.055457 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:32.055544 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 14:58:32.055602 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":67,"threads_started":1}
I20260806 14:58:32.057097 17705 catalog_manager.cc:1347] Generated new cluster ID: c9c5535092954a2f93d531e75552c59d
I20260806 14:58:32.057156 17705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 14:58:32.076229 17705 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 14:58:32.076463 17705 catalog_manager.cc:1504] Loading token signing keys...
I20260806 14:58:32.078584 17705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89b73c91ce6a4efdbbafcf8f943f07d6: Generated new TSK 0
I20260806 14:58:32.078778 17705 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 14:58:32.082345 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:0
--local_ip_for_outbound_sockets=127.11.8.65
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:46685
--builtin_ntp_servers=127.11.8.84:38953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 14:58:32.165342 17720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:32.165544 17720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:32.165565 17720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:32.166746 17720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:32.166785 17720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:32.166811 17720 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 14:58:32.168213 17720 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:38953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:46685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:32.168464 17720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:32.168751 17720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:32.175077 17728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:32.175077 17730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:32.175122 17727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:32.175719 17720 server_base.cc:1034] running on GCE node
I20260806 14:58:32.175952 17720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:32.176196 17720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:32.177395 17720 hybrid_clock.cc:648] HybridClock initialized: now 1786028312177353 us; error 62 us; skew 500 ppm
I20260806 14:58:32.178800 17720 webserver.cc:460] Webserver started at http://127.11.8.65:46567/ using document root <none> and password file <none>
I20260806 14:58:32.179068 17720 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:32.179116 17720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:32.179273 17720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:32.180354 17720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-0/data/instance:
uuid: "0ffaceb62c4142eda6e2adc47ff64b3e"
format_stamp: "Formatted at 2026-08-06 14:58:32 on dist-test-slave-68xk"
server_key: "3409498bbb28ed8bd56a92ecad5f87d8"
server_key_iv: "82b54edf705b8d6a6962f19beb7d7ed2"
server_key_version: "encryptionkey@0"
I20260806 14:58:32.180718 17720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance:
uuid: "0ffaceb62c4142eda6e2adc47ff64b3e"
format_stamp: "Formatted at 2026-08-06 14:58:32 on dist-test-slave-68xk"
server_key: "3409498bbb28ed8bd56a92ecad5f87d8"
server_key_iv: "82b54edf705b8d6a6962f19beb7d7ed2"
server_key_version: "encryptionkey@0"
I20260806 14:58:32.182063 17720 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 14:58:32.182917 17736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:32.183079 17720 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:32.183167 17720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "0ffaceb62c4142eda6e2adc47ff64b3e"
format_stamp: "Formatted at 2026-08-06 14:58:32 on dist-test-slave-68xk"
server_key: "3409498bbb28ed8bd56a92ecad5f87d8"
server_key_iv: "82b54edf705b8d6a6962f19beb7d7ed2"
server_key_version: "encryptionkey@0"
I20260806 14:58:32.183236 17720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:32.186512 17720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:32.186790 17720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:32.186925 17720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:32.187150 17720 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:32.187474 17720 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:32.187505 17720 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:32.187526 17720 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:32.187590 17720 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:32.193364 17720 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:43739
I20260806 14:58:32.193454 17895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:43739 every 8 connection(s)
I20260806 14:58:32.193780 17720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
I20260806 14:58:32.196985 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:32.194613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:53939 (local address 127.11.8.126:46685)
0806 14:58:32.194696 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:32.194700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:32.194952 (+ 252us) server_negotiation.cc:407] Connection header received
0806 14:58:32.195446 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:32.195448 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:32.195456 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:32.195514 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:32.196793 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:32.196794 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:32.196795 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 14:58:32.196799 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:32.196815 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 14:58:32.196820 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:32.196873 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:32.196919 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 14:58:32.196933 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 14:58:32.197186 17900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:32.194598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:46685 (local address 127.11.8.65:53939)
0806 14:58:32.194828 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:32.194862 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:32.195345 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:32.195568 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:32.195571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:32.195580 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:32.196636 (+ 1056us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 14:58:32.196637 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:32.196667 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 14:58:32.196697 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:32.196870 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:32.196872 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:32.196875 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 14:58:32.196925 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 14:58:32.196975 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":123,"threads_started":1}
I20260806 14:58:32.197825 17896 heartbeater.cc:347] Connected to a master server at 127.11.8.126:46685
I20260806 14:58:32.197921 17896 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:32.197947 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 17720
I20260806 14:58:32.198060 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance
I20260806 14:58:32.198099 17896 heartbeater.cc:511] Master 127.11.8.126:46685 requested a full tablet report, sending...
I20260806 14:58:32.198273 11297 external_mini_cluster.cc:1428] Setting key 1e2363a19102c7a1ff40b8c68775adf2
I20260806 14:58:32.198779 17637 ts_manager.cc:194] Registered new tserver with Master: 0ffaceb62c4142eda6e2adc47ff64b3e (127.11.8.65:43739)
I20260806 14:58:32.199466 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:0
--local_ip_for_outbound_sockets=127.11.8.66
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:46685
--builtin_ntp_servers=127.11.8.84:38953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 14:58:32.283810 17901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:32.284024 17901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:32.284056 17901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:32.285271 17901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:32.285317 17901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:32.285343 17901 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 14:58:32.286942 17901 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:38953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:46685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:32.287209 17901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:32.287523 17901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:32.292661 17908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:32.292668 17907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:32.292668 17912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:32.292898 17901 server_base.cc:1034] running on GCE node
I20260806 14:58:32.293135 17901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:32.293391 17901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:32.294541 17901 hybrid_clock.cc:648] HybridClock initialized: now 1786028312294532 us; error 48 us; skew 500 ppm
I20260806 14:58:32.295845 17901 webserver.cc:460] Webserver started at http://127.11.8.66:44465/ using document root <none> and password file <none>
I20260806 14:58:32.296048 17901 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:32.296123 17901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:32.296228 17901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:32.297268 17901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-1/data/instance:
uuid: "1b992c0e797d4884aec43c347c219675"
format_stamp: "Formatted at 2026-08-06 14:58:32 on dist-test-slave-68xk"
server_key: "359cd2adf789ff85a0c29a3aa3975b70"
server_key_iv: "63823a9a3cf320fafe37d81a375f1322"
server_key_version: "encryptionkey@0"
I20260806 14:58:32.297580 17901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance:
uuid: "1b992c0e797d4884aec43c347c219675"
format_stamp: "Formatted at 2026-08-06 14:58:32 on dist-test-slave-68xk"
server_key: "359cd2adf789ff85a0c29a3aa3975b70"
server_key_iv: "63823a9a3cf320fafe37d81a375f1322"
server_key_version: "encryptionkey@0"
I20260806 14:58:32.298959 17901 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 14:58:32.299667 17921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:32.299856 17901 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 14:58:32.299945 17901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "1b992c0e797d4884aec43c347c219675"
format_stamp: "Formatted at 2026-08-06 14:58:32 on dist-test-slave-68xk"
server_key: "359cd2adf789ff85a0c29a3aa3975b70"
server_key_iv: "63823a9a3cf320fafe37d81a375f1322"
server_key_version: "encryptionkey@0"
I20260806 14:58:32.300000 17901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:32.303251 17901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:32.303454 17901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:32.303548 17901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:32.303691 17901 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:32.303938 17901 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:32.303970 17901 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:32.303989 17901 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:32.304003 17901 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:32.309419 17901 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:36783
I20260806 14:58:32.309482 18074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:36783 every 8 connection(s)
I20260806 14:58:32.309823 17901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
I20260806 14:58:32.312994 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:32.310657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:59685 (local address 127.11.8.126:46685)
0806 14:58:32.310728 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:32.310730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:32.310960 (+ 230us) server_negotiation.cc:407] Connection header received
0806 14:58:32.311505 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:32.311506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:32.311514 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:32.311566 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:32.312698 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:32.312701 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:32.312701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 14:58:32.312707 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:32.312725 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 14:58:32.312729 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:32.312806 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:32.312907 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 14:58:32.312923 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 14:58:32.313215 18077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:32.310651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:46685 (local address 127.11.8.66:59685)
0806 14:58:32.310840 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:32.310872 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:32.311383 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:32.311638 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:32.311646 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:32.311654 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:32.312539 (+ 885us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 14:58:32.312540 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:32.312570 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 14:58:32.312599 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:32.312815 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:32.312821 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:32.312824 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 14:58:32.312918 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 14:58:32.312979 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":85,"threads_started":1}
I20260806 14:58:32.313730 18075 heartbeater.cc:347] Connected to a master server at 127.11.8.126:46685
I20260806 14:58:32.313825 18075 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:32.313973 18075 heartbeater.cc:511] Master 127.11.8.126:46685 requested a full tablet report, sending...
I20260806 14:58:32.314350 17637 ts_manager.cc:194] Registered new tserver with Master: 1b992c0e797d4884aec43c347c219675 (127.11.8.66:36783)
I20260806 14:58:32.314900 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 17901
I20260806 14:58:32.314975 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance
I20260806 14:58:32.315106 11297 external_mini_cluster.cc:1428] Setting key 1fb6f887dda3d5af8ae8b01089bd715a
I20260806 14:58:32.316021 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:0
--local_ip_for_outbound_sockets=127.11.8.67
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:46685
--builtin_ntp_servers=127.11.8.84:38953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 14:58:32.400660 18078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:32.400898 18078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:32.400938 18078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:32.402218 18078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:32.402284 18078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:32.402380 18078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 14:58:32.403859 18078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:38953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:46685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:32.404138 18078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:32.404425 18078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:32.409451 18090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:32.409453 18085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:32.409547 18078 server_base.cc:1034] running on GCE node
W20260806 14:58:32.409456 18084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:32.409899 18078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:32.410149 18078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:32.411312 18078 hybrid_clock.cc:648] HybridClock initialized: now 1786028312411297 us; error 47 us; skew 500 ppm
I20260806 14:58:32.412667 18078 webserver.cc:460] Webserver started at http://127.11.8.67:33525/ using document root <none> and password file <none>
I20260806 14:58:32.412926 18078 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:32.412999 18078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:32.413156 18078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:32.414186 18078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-2/data/instance:
uuid: "aa2508ee57bf4da7a1e112dac3f02b7a"
format_stamp: "Formatted at 2026-08-06 14:58:32 on dist-test-slave-68xk"
server_key: "2db97df0d1b0334fe409c2c9dea017e0"
server_key_iv: "d47ee40c15603405f33265903899b37f"
server_key_version: "encryptionkey@0"
I20260806 14:58:32.414584 18078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance:
uuid: "aa2508ee57bf4da7a1e112dac3f02b7a"
format_stamp: "Formatted at 2026-08-06 14:58:32 on dist-test-slave-68xk"
server_key: "2db97df0d1b0334fe409c2c9dea017e0"
server_key_iv: "d47ee40c15603405f33265903899b37f"
server_key_version: "encryptionkey@0"
I20260806 14:58:32.416376 18078 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260806 14:58:32.417091 18102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:32.417285 18078 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 14:58:32.417387 18078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "aa2508ee57bf4da7a1e112dac3f02b7a"
format_stamp: "Formatted at 2026-08-06 14:58:32 on dist-test-slave-68xk"
server_key: "2db97df0d1b0334fe409c2c9dea017e0"
server_key_iv: "d47ee40c15603405f33265903899b37f"
server_key_version: "encryptionkey@0"
I20260806 14:58:32.417474 18078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:32.420799 18078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:32.421069 18078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:32.421213 18078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:32.421430 18078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:32.421727 18078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:32.421788 18078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:32.421828 18078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:32.421855 18078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:32.427817 18078 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:36567
I20260806 14:58:32.427852 18261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:36567 every 8 connection(s)
I20260806 14:58:32.428165 18078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
I20260806 14:58:32.431102 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:32.428921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:60771 (local address 127.11.8.126:46685)
0806 14:58:32.429009 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:32.429011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:32.429180 (+ 169us) server_negotiation.cc:407] Connection header received
0806 14:58:32.429619 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:32.429620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:32.429627 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:32.429681 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:32.430879 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:32.430881 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:32.430881 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 14:58:32.430886 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:32.430902 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 14:58:32.430906 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:32.430963 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:32.431021 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 14:58:32.431049 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 14:58:32.431252 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 18078
I20260806 14:58:32.431318 18267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:32.428866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:46685 (local address 127.11.8.67:60771)
0806 14:58:32.429081 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:32.429111 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:32.429526 (+ 415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:32.429730 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:32.429733 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:32.429742 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:32.430731 (+ 989us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 14:58:32.430731 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:32.430761 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 14:58:32.430789 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:32.430963 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:32.430966 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:32.430969 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 14:58:32.431018 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 14:58:32.431092 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":97,"threads_started":1}
I20260806 14:58:32.431363 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance
I20260806 14:58:32.431552 11297 external_mini_cluster.cc:1428] Setting key 079357dafb9a1965ce23e8e3f48a3dca
I20260806 14:58:32.431896 18263 heartbeater.cc:347] Connected to a master server at 127.11.8.126:46685
I20260806 14:58:32.432042 18263 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:32.432204 18263 heartbeater.cc:511] Master 127.11.8.126:46685 requested a full tablet report, sending...
I20260806 14:58:32.432623 17637 ts_manager.cc:194] Registered new tserver with Master: aa2508ee57bf4da7a1e112dac3f02b7a (127.11.8.67:36567)
I20260806 14:58:32.443410 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 14:58:32.446137 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:32.444851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34878 (local address 127.11.8.126:46685)
0806 14:58:32.444945 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:32.444947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:32.445067 (+ 120us) server_negotiation.cc:407] Connection header received
0806 14:58:32.445160 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:32.445161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:32.445168 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:32.445221 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:32.445916 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:32.445918 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:32.445918 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 14:58:32.445923 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:32.445940 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 14:58:32.445944 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:32.445996 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:32.446044 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 14:58:32.446062 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 14:58:32.447707 17637 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34878:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 14:58:32.449313 17637 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 14:58:32.453398 18275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:32.451612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:43739 (local address 127.0.0.1:37386)
0806 14:58:32.451963 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:32.452004 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:32.452085 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:32.452295 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:32.452302 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:32.452309 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:32.452919 (+ 610us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 14:58:32.452920 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:32.452942 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 14:58:32.452987 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:32.453239 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:32.453246 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:32.453249 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 14:58:32.453300 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 14:58:32.453323 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":86,"threads_started":1}
I20260806 14:58:32.453398 18277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:32.451612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:36567 (local address 127.0.0.1:54146)
0806 14:58:32.452045 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:32.452054 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:32.452090 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:32.452387 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:32.452388 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:32.452390 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:32.453000 (+ 610us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 14:58:32.453001 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:32.453006 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 14:58:32.453015 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:32.453243 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:32.453247 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:32.453249 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 14:58:32.453299 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 14:58:32.453314 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":2,"thread_start_us":74,"threads_started":1}
I20260806 14:58:32.453594 18282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:32.451905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54146 (local address 127.11.8.67:36567)
0806 14:58:32.452169 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:32.452173 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:32.452183 (+ 10us) server_negotiation.cc:407] Connection header received
0806 14:58:32.452208 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:32.452209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:32.452218 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:32.452518 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:32.453106 (+ 588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:32.453111 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:32.453112 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 14:58:32.453116 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:32.453145 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 14:58:32.453168 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:32.453445 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:32.453469 (+ 24us) server_negotiation.cc:299] Negotiation successful
0806 14:58:32.453487 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":106,"threads_started":1}
I20260806 14:58:32.453697 18281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:32.451878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53614 (local address 127.11.8.66:36783)
0806 14:58:32.452217 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:32.452222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:32.452233 (+ 11us) server_negotiation.cc:407] Connection header received
0806 14:58:32.452281 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:32.452284 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:32.452292 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:32.452366 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:32.453276 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:32.453285 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:32.453286 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 14:58:32.453291 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:32.453325 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 14:58:32.453350 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:32.453433 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:32.453581 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 14:58:32.453597 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":81,"threads_started":1}
I20260806 14:58:32.453943 18279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:32.451754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37386 (local address 127.11.8.65:43739)
0806 14:58:32.451992 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:32.451998 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:32.452075 (+ 77us) server_negotiation.cc:407] Connection header received
0806 14:58:32.452160 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:32.452164 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:32.452176 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:32.452520 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:32.453084 (+ 564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:32.453091 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:32.453091 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 14:58:32.453096 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:32.453132 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 14:58:32.453156 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:32.453783 (+ 627us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:32.453806 (+ 23us) server_negotiation.cc:299] Negotiation successful
0806 14:58:32.453824 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":102,"threads_started":1}
I20260806 14:58:32.454041 18276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:32.451661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:36783 (local address 127.0.0.1:53614)
0806 14:58:32.452173 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:32.452182 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:32.452212 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:32.452703 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:32.452704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:32.452707 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:32.453165 (+ 458us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 14:58:32.453166 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:32.453173 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 14:58:32.453187 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:32.453427 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:32.453428 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:32.453429 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 14:58:32.453479 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 14:58:32.453494 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":81,"threads_started":1}
I20260806 14:58:32.454322 18170 tablet_service.cc:1467] Processing CreateTablet for tablet 5a3a9bc3f0ae4646a157413e030a08a5 (DEFAULT_TABLE table=test-table [id=fc822b7f96cf493a9ee99639acca7e2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:32.454727 18170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a3a9bc3f0ae4646a157413e030a08a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:32.454710 17803 tablet_service.cc:1467] Processing CreateTablet for tablet 5a3a9bc3f0ae4646a157413e030a08a5 (DEFAULT_TABLE table=test-table [id=fc822b7f96cf493a9ee99639acca7e2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:32.454754 17987 tablet_service.cc:1467] Processing CreateTablet for tablet 5a3a9bc3f0ae4646a157413e030a08a5 (DEFAULT_TABLE table=test-table [id=fc822b7f96cf493a9ee99639acca7e2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:32.455092 17987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a3a9bc3f0ae4646a157413e030a08a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:32.455101 17803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a3a9bc3f0ae4646a157413e030a08a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:32.457286 18283 tablet_bootstrap.cc:492] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a: Bootstrap starting.
I20260806 14:58:32.458002 18283 tablet_bootstrap.cc:654] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:32.458326 18285 tablet_bootstrap.cc:492] T 5a3a9bc3f0ae4646a157413e030a08a5 P 1b992c0e797d4884aec43c347c219675: Bootstrap starting.
I20260806 14:58:32.458487 18283 log.cc:824] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:32.459007 18285 tablet_bootstrap.cc:654] T 5a3a9bc3f0ae4646a157413e030a08a5 P 1b992c0e797d4884aec43c347c219675: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:32.459288 18283 tablet_bootstrap.cc:492] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a: No bootstrap required, opened a new log
I20260806 14:58:32.459344 18285 log.cc:824] T 5a3a9bc3f0ae4646a157413e030a08a5 P 1b992c0e797d4884aec43c347c219675: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:32.459364 18283 ts_tablet_manager.cc:1397] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 14:58:32.459679 18286 tablet_bootstrap.cc:492] T 5a3a9bc3f0ae4646a157413e030a08a5 P 0ffaceb62c4142eda6e2adc47ff64b3e: Bootstrap starting.
I20260806 14:58:32.460090 18285 tablet_bootstrap.cc:492] T 5a3a9bc3f0ae4646a157413e030a08a5 P 1b992c0e797d4884aec43c347c219675: No bootstrap required, opened a new log
I20260806 14:58:32.460167 18285 ts_tablet_manager.cc:1397] T 5a3a9bc3f0ae4646a157413e030a08a5 P 1b992c0e797d4884aec43c347c219675: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 14:58:32.460479 18286 tablet_bootstrap.cc:654] T 5a3a9bc3f0ae4646a157413e030a08a5 P 0ffaceb62c4142eda6e2adc47ff64b3e: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:32.460798 18286 log.cc:824] T 5a3a9bc3f0ae4646a157413e030a08a5 P 0ffaceb62c4142eda6e2adc47ff64b3e: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:32.461617 18286 tablet_bootstrap.cc:492] T 5a3a9bc3f0ae4646a157413e030a08a5 P 0ffaceb62c4142eda6e2adc47ff64b3e: No bootstrap required, opened a new log
I20260806 14:58:32.461700 18286 ts_tablet_manager.cc:1397] T 5a3a9bc3f0ae4646a157413e030a08a5 P 0ffaceb62c4142eda6e2adc47ff64b3e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 14:58:32.462064 18283 raft_consensus.cc:348] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2508ee57bf4da7a1e112dac3f02b7a" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36567 } } peers { permanent_uuid: "0ffaceb62c4142eda6e2adc47ff64b3e" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43739 } } peers { permanent_uuid: "1b992c0e797d4884aec43c347c219675" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36783 } }
I20260806 14:58:32.462234 18283 raft_consensus.cc:374] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:32.462270 18283 raft_consensus.cc:729] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa2508ee57bf4da7a1e112dac3f02b7a, State: Initialized, Role: FOLLOWER
I20260806 14:58:32.462435 18283 consensus_queue.cc:260] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2508ee57bf4da7a1e112dac3f02b7a" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36567 } } peers { permanent_uuid: "0ffaceb62c4142eda6e2adc47ff64b3e" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43739 } } peers { permanent_uuid: "1b992c0e797d4884aec43c347c219675" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36783 } }
I20260806 14:58:32.462370 18285 raft_consensus.cc:348] T 5a3a9bc3f0ae4646a157413e030a08a5 P 1b992c0e797d4884aec43c347c219675 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2508ee57bf4da7a1e112dac3f02b7a" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36567 } } peers { permanent_uuid: "0ffaceb62c4142eda6e2adc47ff64b3e" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43739 } } peers { permanent_uuid: "1b992c0e797d4884aec43c347c219675" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36783 } }
I20260806 14:58:32.462594 18285 raft_consensus.cc:374] T 5a3a9bc3f0ae4646a157413e030a08a5 P 1b992c0e797d4884aec43c347c219675 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:32.462637 18285 raft_consensus.cc:729] T 5a3a9bc3f0ae4646a157413e030a08a5 P 1b992c0e797d4884aec43c347c219675 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b992c0e797d4884aec43c347c219675, State: Initialized, Role: FOLLOWER
I20260806 14:58:32.462697 18283 ts_tablet_manager.cc:1428] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 14:58:32.462829 18263 heartbeater.cc:503] Master 127.11.8.126:46685 was elected leader, sending a full tablet report...
I20260806 14:58:32.462775 18285 consensus_queue.cc:260] T 5a3a9bc3f0ae4646a157413e030a08a5 P 1b992c0e797d4884aec43c347c219675 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2508ee57bf4da7a1e112dac3f02b7a" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36567 } } peers { permanent_uuid: "0ffaceb62c4142eda6e2adc47ff64b3e" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43739 } } peers { permanent_uuid: "1b992c0e797d4884aec43c347c219675" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36783 } }
I20260806 14:58:32.463368 18075 heartbeater.cc:503] Master 127.11.8.126:46685 was elected leader, sending a full tablet report...
I20260806 14:58:32.463353 18285 ts_tablet_manager.cc:1428] T 5a3a9bc3f0ae4646a157413e030a08a5 P 1b992c0e797d4884aec43c347c219675: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 14:58:32.464326 18286 raft_consensus.cc:348] T 5a3a9bc3f0ae4646a157413e030a08a5 P 0ffaceb62c4142eda6e2adc47ff64b3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2508ee57bf4da7a1e112dac3f02b7a" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36567 } } peers { permanent_uuid: "0ffaceb62c4142eda6e2adc47ff64b3e" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43739 } } peers { permanent_uuid: "1b992c0e797d4884aec43c347c219675" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36783 } }
I20260806 14:58:32.464474 18286 raft_consensus.cc:374] T 5a3a9bc3f0ae4646a157413e030a08a5 P 0ffaceb62c4142eda6e2adc47ff64b3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:32.464527 18286 raft_consensus.cc:729] T 5a3a9bc3f0ae4646a157413e030a08a5 P 0ffaceb62c4142eda6e2adc47ff64b3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ffaceb62c4142eda6e2adc47ff64b3e, State: Initialized, Role: FOLLOWER
I20260806 14:58:32.464637 18286 consensus_queue.cc:260] T 5a3a9bc3f0ae4646a157413e030a08a5 P 0ffaceb62c4142eda6e2adc47ff64b3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2508ee57bf4da7a1e112dac3f02b7a" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36567 } } peers { permanent_uuid: "0ffaceb62c4142eda6e2adc47ff64b3e" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43739 } } peers { permanent_uuid: "1b992c0e797d4884aec43c347c219675" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36783 } }
I20260806 14:58:32.464870 18286 ts_tablet_manager.cc:1428] T 5a3a9bc3f0ae4646a157413e030a08a5 P 0ffaceb62c4142eda6e2adc47ff64b3e: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 14:58:32.464911 17896 heartbeater.cc:503] Master 127.11.8.126:46685 was elected leader, sending a full tablet report...
W20260806 14:58:32.560427 18076 tablet.cc:2369] T 5a3a9bc3f0ae4646a157413e030a08a5 P 1b992c0e797d4884aec43c347c219675: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 14:58:32.590998 18291 raft_consensus.cc:482] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 14:58:32.591153 18291 raft_consensus.cc:504] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2508ee57bf4da7a1e112dac3f02b7a" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36567 } } peers { permanent_uuid: "0ffaceb62c4142eda6e2adc47ff64b3e" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43739 } } peers { permanent_uuid: "1b992c0e797d4884aec43c347c219675" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36783 } }
I20260806 14:58:32.591562 18291 leader_election.cc:288] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ffaceb62c4142eda6e2adc47ff64b3e (127.11.8.65:43739), 1b992c0e797d4884aec43c347c219675 (127.11.8.66:36783)
I20260806 14:58:32.593000 18267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:32.591797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:43739 (local address 127.11.8.67:48041)
0806 14:58:32.591874 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:32.591887 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:32.591951 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:32.592135 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:32.592136 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:32.592139 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:32.592650 (+ 511us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 14:58:32.592650 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:32.592657 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 14:58:32.592669 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:32.592872 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:32.592873 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:32.592874 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 14:58:32.592915 (+ 41us) client_negotiation.cc:240] Negotiation successful
0806 14:58:32.592931 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 14:58:32.593003 18279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:32.591874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:48041 (local address 127.11.8.65:43739)
0806 14:58:32.591921 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:32.591923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:32.591937 (+ 14us) server_negotiation.cc:407] Connection header received
0806 14:58:32.592044 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:32.592046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:32.592054 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:32.592114 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:32.592765 (+ 651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:32.592766 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:32.592766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 14:58:32.592771 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:32.592794 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 14:58:32.592799 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:32.592861 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:32.592926 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 14:58:32.592940 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 14:58:32.593178 18295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:32.591797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:36783 (local address 127.11.8.67:50871)
0806 14:58:32.591999 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:32.592011 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:32.592073 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:32.592224 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:32.592225 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:32.592227 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:32.592810 (+ 583us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 14:58:32.592811 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:32.592818 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 14:58:32.592832 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:32.593023 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:32.593024 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:32.593025 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 14:58:32.593075 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 14:58:32.593086 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":21,"thread_start_us":93,"threads_started":1}
I20260806 14:58:32.593166 18281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:32.591905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:50871 (local address 127.11.8.66:36783)
0806 14:58:32.591991 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:32.591993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:32.592079 (+ 86us) server_negotiation.cc:407] Connection header received
0806 14:58:32.592145 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:32.592146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:32.592152 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:32.592214 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:32.592932 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:32.592933 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:32.592934 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 14:58:32.592939 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:32.592957 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 14:58:32.592961 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:32.593021 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:32.593070 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 14:58:32.593086 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 14:58:32.593559 17832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3a9bc3f0ae4646a157413e030a08a5" candidate_uuid: "aa2508ee57bf4da7a1e112dac3f02b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffaceb62c4142eda6e2adc47ff64b3e" is_pre_election: true
I20260806 14:58:32.593560 18013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3a9bc3f0ae4646a157413e030a08a5" candidate_uuid: "aa2508ee57bf4da7a1e112dac3f02b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b992c0e797d4884aec43c347c219675" is_pre_election: true
I20260806 14:58:32.593715 18013 raft_consensus.cc:2454] T 5a3a9bc3f0ae4646a157413e030a08a5 P 1b992c0e797d4884aec43c347c219675 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa2508ee57bf4da7a1e112dac3f02b7a in term 0.
I20260806 14:58:32.593715 17832 raft_consensus.cc:2454] T 5a3a9bc3f0ae4646a157413e030a08a5 P 0ffaceb62c4142eda6e2adc47ff64b3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa2508ee57bf4da7a1e112dac3f02b7a in term 0.
I20260806 14:58:32.594055 18103 leader_election.cc:302] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b992c0e797d4884aec43c347c219675, aa2508ee57bf4da7a1e112dac3f02b7a; no voters:
I20260806 14:58:32.594262 18291 raft_consensus.cc:2781] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 14:58:32.594321 18291 raft_consensus.cc:482] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 14:58:32.594350 18291 raft_consensus.cc:3037] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:32.595330 18291 raft_consensus.cc:504] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2508ee57bf4da7a1e112dac3f02b7a" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36567 } } peers { permanent_uuid: "0ffaceb62c4142eda6e2adc47ff64b3e" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43739 } } peers { permanent_uuid: "1b992c0e797d4884aec43c347c219675" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36783 } }
I20260806 14:58:32.595499 18291 leader_election.cc:288] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a [CANDIDATE]: Term 1 election: Requested vote from peers 0ffaceb62c4142eda6e2adc47ff64b3e (127.11.8.65:43739), 1b992c0e797d4884aec43c347c219675 (127.11.8.66:36783)
I20260806 14:58:32.595700 18013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3a9bc3f0ae4646a157413e030a08a5" candidate_uuid: "aa2508ee57bf4da7a1e112dac3f02b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b992c0e797d4884aec43c347c219675"
I20260806 14:58:32.595790 18013 raft_consensus.cc:3037] T 5a3a9bc3f0ae4646a157413e030a08a5 P 1b992c0e797d4884aec43c347c219675 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:32.596577 17832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3a9bc3f0ae4646a157413e030a08a5" candidate_uuid: "aa2508ee57bf4da7a1e112dac3f02b7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ffaceb62c4142eda6e2adc47ff64b3e"
I20260806 14:58:32.596701 17832 raft_consensus.cc:3037] T 5a3a9bc3f0ae4646a157413e030a08a5 P 0ffaceb62c4142eda6e2adc47ff64b3e [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:32.596848 18013 raft_consensus.cc:2454] T 5a3a9bc3f0ae4646a157413e030a08a5 P 1b992c0e797d4884aec43c347c219675 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa2508ee57bf4da7a1e112dac3f02b7a in term 1.
I20260806 14:58:32.597479 17832 raft_consensus.cc:2454] T 5a3a9bc3f0ae4646a157413e030a08a5 P 0ffaceb62c4142eda6e2adc47ff64b3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa2508ee57bf4da7a1e112dac3f02b7a in term 1.
I20260806 14:58:32.597667 18105 leader_election.cc:302] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ffaceb62c4142eda6e2adc47ff64b3e, aa2508ee57bf4da7a1e112dac3f02b7a; no voters:
I20260806 14:58:32.597792 18291 raft_consensus.cc:2781] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:32.597949 18291 raft_consensus.cc:686] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a [term 1 LEADER]: Becoming Leader. State: Replica: aa2508ee57bf4da7a1e112dac3f02b7a, State: Running, Role: LEADER
I20260806 14:58:32.598066 18291 consensus_queue.cc:237] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2508ee57bf4da7a1e112dac3f02b7a" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36567 } } peers { permanent_uuid: "0ffaceb62c4142eda6e2adc47ff64b3e" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43739 } } peers { permanent_uuid: "1b992c0e797d4884aec43c347c219675" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36783 } }
I20260806 14:58:32.599066 17634 catalog_manager.cc:5495] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a reported cstate change: term changed from 0 to 1, leader changed from <none> to aa2508ee57bf4da7a1e112dac3f02b7a (127.11.8.67). New cstate: current_term: 1 leader_uuid: "aa2508ee57bf4da7a1e112dac3f02b7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa2508ee57bf4da7a1e112dac3f02b7a" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 36567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ffaceb62c4142eda6e2adc47ff64b3e" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b992c0e797d4884aec43c347c219675" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36783 } health_report { overall_health: UNKNOWN } } }
I20260806 14:58:32.638703 18282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:32.637601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54162 (local address 127.11.8.67:36567)
0806 14:58:32.637692 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:32.637694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:32.637703 (+ 9us) server_negotiation.cc:407] Connection header received
0806 14:58:32.637740 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:32.637741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:32.637756 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:32.637813 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:32.638452 (+ 639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:32.638453 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:32.638454 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 14:58:32.638458 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:32.638473 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 14:58:32.638477 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:32.638530 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:32.638581 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 14:58:32.638592 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 14:58:32.640661 17832 raft_consensus.cc:1261] T 5a3a9bc3f0ae4646a157413e030a08a5 P 0ffaceb62c4142eda6e2adc47ff64b3e [term 1 FOLLOWER]: Refusing update from remote peer aa2508ee57bf4da7a1e112dac3f02b7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:32.640661 18013 raft_consensus.cc:1261] T 5a3a9bc3f0ae4646a157413e030a08a5 P 1b992c0e797d4884aec43c347c219675 [term 1 FOLLOWER]: Refusing update from remote peer aa2508ee57bf4da7a1e112dac3f02b7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:32.641011 18291 consensus_queue.cc:1035] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ffaceb62c4142eda6e2adc47ff64b3e" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:32.641197 18291 consensus_queue.cc:1035] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b992c0e797d4884aec43c347c219675" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:32.644265 18302 mvcc.cc:205] Tried to move back new op lower bound from 7315571968573435904 to 7315571968402472960. Current Snapshot: MvccSnapshot[applied={T|T < 7315571968573435904}]
I20260806 14:58:32.646466 17634 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:58:32.646580 17634 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34878:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:58:32.647423 17634 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 89b73c91ce6a4efdbbafcf8f943f07d6: Sending DeleteTablet for 3 replicas of tablet 5a3a9bc3f0ae4646a157413e030a08a5
I20260806 14:58:32.647763 18170 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a3a9bc3f0ae4646a157413e030a08a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:32 UTC) from {username='slave'} at 127.0.0.1:54146
I20260806 14:58:32.647792 17803 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a3a9bc3f0ae4646a157413e030a08a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:32 UTC) from {username='slave'} at 127.0.0.1:37386
I20260806 14:58:32.647917 17987 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a3a9bc3f0ae4646a157413e030a08a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:32 UTC) from {username='slave'} at 127.0.0.1:53614
I20260806 14:58:32.648082 18314 tablet_replica.cc:330] T 5a3a9bc3f0ae4646a157413e030a08a5 P 0ffaceb62c4142eda6e2adc47ff64b3e: stopping tablet replica
I20260806 14:58:32.648157 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 17720
I20260806 14:58:32.648183 18314 raft_consensus.cc:2229] T 5a3a9bc3f0ae4646a157413e030a08a5 P 0ffaceb62c4142eda6e2adc47ff64b3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:58:32.654304 18315 tablet_replica.cc:330] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a: stopping tablet replica
I20260806 14:58:32.654428 18315 raft_consensus.cc:2229] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a [term 1 LEADER]: Raft consensus shutting down.
I20260806 14:58:32.654563 18315 raft_consensus.cc:2258] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:32.655090 18315 ts_tablet_manager.cc:1905] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:32.656988 18315 ts_tablet_manager.cc:1918] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 14:58:32.657068 18315 log.cc:1196] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786028287744255-11297-0/minicluster-data/ts-2/wal/wals/5a3a9bc3f0ae4646a157413e030a08a5
I20260806 14:58:32.657377 18315 ts_tablet_manager.cc:1939] T 5a3a9bc3f0ae4646a157413e030a08a5 P aa2508ee57bf4da7a1e112dac3f02b7a: Deleting consensus metadata
I20260806 14:58:32.657821 17621 catalog_manager.cc:4841] TS aa2508ee57bf4da7a1e112dac3f02b7a (127.11.8.67:36567): tablet 5a3a9bc3f0ae4646a157413e030a08a5 (table test-table [id=fc822b7f96cf493a9ee99639acca7e2a]) successfully deleted
W20260806 14:58:32.660804 17620 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.65:43739 (error 108)
I20260806 14:58:32.661388 18276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:32.661227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:43739 (local address 127.0.0.1:37396)
0806 14:58:32.661301 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:32.661348 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.65:43739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 14:58:32.661458 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 17901
W20260806 14:58:32.666608 17620 catalog_manager.cc:4584] TS 0ffaceb62c4142eda6e2adc47ff64b3e (127.11.8.65:43739): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5a3a9bc3f0ae4646a157413e030a08a5: Network error: Client connection negotiation failed: client connection to 127.11.8.65:43739: connect: Connection refused (error 111)
I20260806 14:58:32.669987 18276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:32.669858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:36783 (local address 127.0.0.1:53622)
0806 14:58:32.669920 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:32.669952 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.66:36783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 14:58:32.669984 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 18078
I20260806 14:58:32.679622 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 17595
2026-08-06T14:58:32Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (774 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 14:58:32.704053 11297 mini_oidc.cc:152] Starting JWKS server
I20260806 14:58:32.705284 11297 webserver.cc:460] Webserver started at https://127.0.0.1:33173/ using document root <none> and password file <none>
I20260806 14:58:32.705368 11297 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 14:58:32.705500 11297 webserver.cc:460] Webserver started at http://127.0.0.1:46793/ using document root <none> and password file <none>
2026-08-06T14:58:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T14:58:32Z Disabled control of system clock
I20260806 14:58:32.714713 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:33531
--webserver_interface=127.11.8.126
--webserver_port=0
--builtin_ntp_servers=127.11.8.84:38865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:33531
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33173/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028287744255-11297-0/testchainca.cert with env {}
W20260806 14:58:32.797248 18325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:32.797457 18325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:32.797479 18325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:32.798602 18325 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 14:58:32.798643 18325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:32.798656 18325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:32.798671 18325 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 14:58:32.798682 18325 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 14:58:32.798695 18325 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 14:58:32.798707 18325 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33173/jwks.json
I20260806 14:58:32.800209 18325 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:38865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:33531
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:33531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33173/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028287744255-11297-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:32.800403 18325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:32.800570 18325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:32.805416 18336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:32.805393 18332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:32.805394 18333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:32.805667 18325 server_base.cc:1034] running on GCE node
I20260806 14:58:32.805881 18325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:32.806126 18325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:32.807313 18325 hybrid_clock.cc:648] HybridClock initialized: now 1786028312807267 us; error 69 us; skew 500 ppm
I20260806 14:58:32.808799 18325 webserver.cc:460] Webserver started at http://127.11.8.126:40135/ using document root <none> and password file <none>
I20260806 14:58:32.809072 18325 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:32.809125 18325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:32.809297 18325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:32.810527 18325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "4b5f09ce16344aaca8c0df98d0af7e51"
format_stamp: "Formatted at 2026-08-06 14:58:32 on dist-test-slave-68xk"
server_key: "4c0eb57beb4bc9109e5695be29ff1410"
server_key_iv: "e2fed1ee6772438f5db4bb83f7babb8a"
server_key_version: "encryptionkey@0"
I20260806 14:58:32.810925 18325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "4b5f09ce16344aaca8c0df98d0af7e51"
format_stamp: "Formatted at 2026-08-06 14:58:32 on dist-test-slave-68xk"
server_key: "4c0eb57beb4bc9109e5695be29ff1410"
server_key_iv: "e2fed1ee6772438f5db4bb83f7babb8a"
server_key_version: "encryptionkey@0"
I20260806 14:58:32.812330 18325 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 14:58:32.813128 18345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:32.813419 18325 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 14:58:32.813536 18325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "4b5f09ce16344aaca8c0df98d0af7e51"
format_stamp: "Formatted at 2026-08-06 14:58:32 on dist-test-slave-68xk"
server_key: "4c0eb57beb4bc9109e5695be29ff1410"
server_key_iv: "e2fed1ee6772438f5db4bb83f7babb8a"
server_key_version: "encryptionkey@0"
I20260806 14:58:32.813618 18325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:32.827239 18325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:32.827603 18325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:32.827795 18325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:32.832523 18325 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:33531
I20260806 14:58:32.832580 18408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:33531 every 8 connection(s)
I20260806 14:58:32.832967 18325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 14:58:32.834009 18409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:32.836360 18409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b5f09ce16344aaca8c0df98d0af7e51: Bootstrap starting.
I20260806 14:58:32.836977 18409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b5f09ce16344aaca8c0df98d0af7e51: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:32.837298 18409 log.cc:824] T 00000000000000000000000000000000 P 4b5f09ce16344aaca8c0df98d0af7e51: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:32.838037 18409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b5f09ce16344aaca8c0df98d0af7e51: No bootstrap required, opened a new log
I20260806 14:58:32.839161 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 18325
I20260806 14:58:32.839293 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 14:58:32.839505 11297 external_mini_cluster.cc:1428] Setting key 66249f51c161e33ab47cbf9403d53e3a
I20260806 14:58:32.839874 18409 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b5f09ce16344aaca8c0df98d0af7e51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5f09ce16344aaca8c0df98d0af7e51" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 33531 } }
I20260806 14:58:32.840068 18409 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b5f09ce16344aaca8c0df98d0af7e51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:32.840118 18409 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b5f09ce16344aaca8c0df98d0af7e51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b5f09ce16344aaca8c0df98d0af7e51, State: Initialized, Role: FOLLOWER
I20260806 14:58:32.840296 18409 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b5f09ce16344aaca8c0df98d0af7e51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5f09ce16344aaca8c0df98d0af7e51" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 33531 } }
I20260806 14:58:32.840394 18409 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b5f09ce16344aaca8c0df98d0af7e51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 14:58:32.840435 18409 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b5f09ce16344aaca8c0df98d0af7e51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 14:58:32.840488 18409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b5f09ce16344aaca8c0df98d0af7e51 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:32.841210 18409 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b5f09ce16344aaca8c0df98d0af7e51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5f09ce16344aaca8c0df98d0af7e51" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 33531 } }
I20260806 14:58:32.841342 18409 leader_election.cc:302] T 00000000000000000000000000000000 P 4b5f09ce16344aaca8c0df98d0af7e51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b5f09ce16344aaca8c0df98d0af7e51; no voters:
I20260806 14:58:32.841529 18409 leader_election.cc:288] T 00000000000000000000000000000000 P 4b5f09ce16344aaca8c0df98d0af7e51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 14:58:32.841600 18420 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b5f09ce16344aaca8c0df98d0af7e51 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:32.841763 18420 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b5f09ce16344aaca8c0df98d0af7e51 [term 1 LEADER]: Becoming Leader. State: Replica: 4b5f09ce16344aaca8c0df98d0af7e51, State: Running, Role: LEADER
I20260806 14:58:32.841812 18409 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b5f09ce16344aaca8c0df98d0af7e51 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 14:58:32.841874 18420 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b5f09ce16344aaca8c0df98d0af7e51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5f09ce16344aaca8c0df98d0af7e51" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 33531 } }
I20260806 14:58:32.842236 18420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b5f09ce16344aaca8c0df98d0af7e51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b5f09ce16344aaca8c0df98d0af7e51. Latest consensus state: current_term: 1 leader_uuid: "4b5f09ce16344aaca8c0df98d0af7e51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5f09ce16344aaca8c0df98d0af7e51" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 33531 } } }
I20260806 14:58:32.842319 18420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b5f09ce16344aaca8c0df98d0af7e51 [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:32.842245 18421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b5f09ce16344aaca8c0df98d0af7e51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b5f09ce16344aaca8c0df98d0af7e51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5f09ce16344aaca8c0df98d0af7e51" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 33531 } } }
I20260806 14:58:32.842365 18421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b5f09ce16344aaca8c0df98d0af7e51 [sys.catalog]: This master's current role is: LEADER
W20260806 14:58:32.843858 18436 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4b5f09ce16344aaca8c0df98d0af7e51: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 14:58:32.843924 18436 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 14:58:32.843971 18428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 14:58:32.844120 18428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 14:58:32.845867 18428 catalog_manager.cc:1347] Generated new cluster ID: a747eb43ac0b4a9ab5b3e3220de9e5e6
I20260806 14:58:32.845930 18428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 14:58:32.846361 18418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:32.840094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38592 (local address 127.11.8.126:33531)
0806 14:58:32.840348 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:32.840356 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:32.840382 (+ 26us) server_negotiation.cc:407] Connection header received
0806 14:58:32.840531 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:32.840553 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:32.840633 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:32.840779 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:32.841052 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:32.842953 (+ 1901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:32.844249 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:32.844675 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:32.844820 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:32.845613 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:32.845617 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:32.845617 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 14:58:32.845829 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:32.845873 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 14:58:32.845888 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:32.845954 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:32.846068 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 14:58:32.846146 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":107,"threads_started":1}
I20260806 14:58:32.847337 11297 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 14:58:32.864163 18428 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 14:58:32.864869 18428 catalog_manager.cc:1504] Loading token signing keys...
I20260806 14:58:32.877724 18428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b5f09ce16344aaca8c0df98d0af7e51: Generated new TSK 0
I20260806 14:58:32.877931 18428 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 14:58:32.909677 18447 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 14:58:32.909829 18447 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 14:58:32.909919 18418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:32.887112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38608 (local address 127.11.8.126:33531)
0806 14:58:32.887182 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:32.887184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:32.887339 (+ 155us) server_negotiation.cc:407] Connection header received
0806 14:58:32.887449 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:32.887450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:32.887463 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:32.887546 (+ 83us) server_negotiation.cc:226] Negotiated authn=JWT
0806 14:58:32.887704 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:32.888190 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:32.888959 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:32.889168 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:32.889253 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:32.889458 (+ 205us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 14:58:32.909639 (+ 20181us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33173/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 14:58:32.909843 (+ 204us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38608: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33173/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 14:58:32.909987 18418 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38608: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33173/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 14:58:32.910581 18446 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:32.887046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:33531 (local address 127.0.0.1:38608)
0806 14:58:32.887273 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:32.887282 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:32.887363 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:32.887555 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:32.887556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:32.887558 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 14:58:32.887623 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:32.887625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:32.888287 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:32.888288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:32.888870 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:32.888872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:32.889249 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:32.889250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:32.889315 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:32.889338 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 14:58:32.910066 (+ 20728us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33173/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 14:58:32.910207 (+ 141us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.8.126:33531: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33173/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":103,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.11.8.126:33531: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33173/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 14:58:32.911147 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 18325
2026-08-06T14:58:32Z chronyd exiting
I20260806 14:58:32.953812 11297 test_util.cc:175] -----------------------------------------------
I20260806 14:58:32.953914 11297 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786028287744255-11297-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (250 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 14:58:32.954857 11297 mini_oidc.cc:152] Starting JWKS server
I20260806 14:58:32.955551 11297 webserver.cc:460] Webserver started at https://127.0.0.1:45287/ using document root <none> and password file <none>
I20260806 14:58:32.955619 11297 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 14:58:32.955806 11297 webserver.cc:460] Webserver started at http://127.0.0.1:38577/ using document root <none> and password file <none>
2026-08-06T14:58:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T14:58:32Z Disabled control of system clock
I20260806 14:58:32.964334 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:44937
--webserver_interface=127.11.8.126
--webserver_port=0
--builtin_ntp_servers=127.11.8.84:45995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:44937
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45287/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028287744255-11297-0/testchainca.cert with env {}
W20260806 14:58:33.049016 18456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:33.049242 18456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:33.049283 18456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:33.050508 18456 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 14:58:33.050567 18456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:33.050587 18456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:33.050609 18456 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 14:58:33.050638 18456 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 14:58:33.050666 18456 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 14:58:33.050695 18456 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45287/jwks.json
I20260806 14:58:33.052209 18456 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:45995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:44937
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:44937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45287/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028287744255-11297-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:33.052486 18456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:33.052703 18456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:33.057523 18461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:33.057539 18462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:33.057680 18456 server_base.cc:1034] running on GCE node
W20260806 14:58:33.057519 18465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:33.057972 18456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:33.058264 18456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:33.059428 18456 hybrid_clock.cc:648] HybridClock initialized: now 1786028313059413 us; error 31 us; skew 500 ppm
I20260806 14:58:33.060598 18456 webserver.cc:460] Webserver started at http://127.11.8.126:40107/ using document root <none> and password file <none>
I20260806 14:58:33.060856 18456 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:33.060928 18456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:33.061077 18456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:33.062062 18456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "091dac5445d948a1ac412b559724fdbe"
format_stamp: "Formatted at 2026-08-06 14:58:33 on dist-test-slave-68xk"
server_key: "0e5393f99736c6d3af7e193c70778152"
server_key_iv: "c903c6c3f195d603e935b243746ba7d4"
server_key_version: "encryptionkey@0"
I20260806 14:58:33.062465 18456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "091dac5445d948a1ac412b559724fdbe"
format_stamp: "Formatted at 2026-08-06 14:58:33 on dist-test-slave-68xk"
server_key: "0e5393f99736c6d3af7e193c70778152"
server_key_iv: "c903c6c3f195d603e935b243746ba7d4"
server_key_version: "encryptionkey@0"
I20260806 14:58:33.063755 18456 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 14:58:33.064472 18471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:33.064649 18456 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 14:58:33.064749 18456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "091dac5445d948a1ac412b559724fdbe"
format_stamp: "Formatted at 2026-08-06 14:58:33 on dist-test-slave-68xk"
server_key: "0e5393f99736c6d3af7e193c70778152"
server_key_iv: "c903c6c3f195d603e935b243746ba7d4"
server_key_version: "encryptionkey@0"
I20260806 14:58:33.064833 18456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:33.093423 18456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:33.093753 18456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:33.093916 18456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:33.097990 18456 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:44937
I20260806 14:58:33.098119 18534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:44937 every 8 connection(s)
I20260806 14:58:33.098445 18456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 14:58:33.098948 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 18456
I20260806 14:58:33.099123 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 14:58:33.099370 11297 external_mini_cluster.cc:1428] Setting key 2479b9d3bd1cecf9855433165a5dab78
I20260806 14:58:33.099370 18535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:33.102586 18535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 091dac5445d948a1ac412b559724fdbe: Bootstrap starting.
I20260806 14:58:33.103331 18535 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 091dac5445d948a1ac412b559724fdbe: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:33.103641 18535 log.cc:824] T 00000000000000000000000000000000 P 091dac5445d948a1ac412b559724fdbe: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:33.104427 18535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 091dac5445d948a1ac412b559724fdbe: No bootstrap required, opened a new log
I20260806 14:58:33.105443 18538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:33.100110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52874 (local address 127.11.8.126:44937)
0806 14:58:33.100313 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:33.100322 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:33.100355 (+ 33us) server_negotiation.cc:407] Connection header received
0806 14:58:33.100817 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:33.100824 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:33.100884 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:33.101000 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:33.101296 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:33.102160 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:33.103287 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:33.103825 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:33.103967 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:33.104725 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:33.104729 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:33.104729 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 14:58:33.104895 (+ 166us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:33.104932 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 14:58:33.104942 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:33.104994 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:33.105121 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 14:58:33.105204 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260806 14:58:33.106601 11297 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 14:58:33.106871 18535 raft_consensus.cc:348] T 00000000000000000000000000000000 P 091dac5445d948a1ac412b559724fdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091dac5445d948a1ac412b559724fdbe" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44937 } }
I20260806 14:58:33.107035 18535 raft_consensus.cc:374] T 00000000000000000000000000000000 P 091dac5445d948a1ac412b559724fdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:33.107086 18535 raft_consensus.cc:729] T 00000000000000000000000000000000 P 091dac5445d948a1ac412b559724fdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 091dac5445d948a1ac412b559724fdbe, State: Initialized, Role: FOLLOWER
I20260806 14:58:33.107211 18535 consensus_queue.cc:260] T 00000000000000000000000000000000 P 091dac5445d948a1ac412b559724fdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091dac5445d948a1ac412b559724fdbe" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44937 } }
I20260806 14:58:33.107290 18535 raft_consensus.cc:388] T 00000000000000000000000000000000 P 091dac5445d948a1ac412b559724fdbe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 14:58:33.107326 18535 raft_consensus.cc:482] T 00000000000000000000000000000000 P 091dac5445d948a1ac412b559724fdbe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 14:58:33.107374 18535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 091dac5445d948a1ac412b559724fdbe [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:33.107996 18535 raft_consensus.cc:504] T 00000000000000000000000000000000 P 091dac5445d948a1ac412b559724fdbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091dac5445d948a1ac412b559724fdbe" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44937 } }
I20260806 14:58:33.108091 18535 leader_election.cc:302] T 00000000000000000000000000000000 P 091dac5445d948a1ac412b559724fdbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 091dac5445d948a1ac412b559724fdbe; no voters:
I20260806 14:58:33.108222 18535 leader_election.cc:288] T 00000000000000000000000000000000 P 091dac5445d948a1ac412b559724fdbe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 14:58:33.108300 18540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 091dac5445d948a1ac412b559724fdbe [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:33.108458 18540 raft_consensus.cc:686] T 00000000000000000000000000000000 P 091dac5445d948a1ac412b559724fdbe [term 1 LEADER]: Becoming Leader. State: Replica: 091dac5445d948a1ac412b559724fdbe, State: Running, Role: LEADER
I20260806 14:58:33.108512 18535 sys_catalog.cc:564] T 00000000000000000000000000000000 P 091dac5445d948a1ac412b559724fdbe [sys.catalog]: configured and running, proceeding with master startup.
I20260806 14:58:33.108549 18540 consensus_queue.cc:237] T 00000000000000000000000000000000 P 091dac5445d948a1ac412b559724fdbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091dac5445d948a1ac412b559724fdbe" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44937 } }
I20260806 14:58:33.108917 18541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 091dac5445d948a1ac412b559724fdbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "091dac5445d948a1ac412b559724fdbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091dac5445d948a1ac412b559724fdbe" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44937 } } }
I20260806 14:58:33.109012 18541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 091dac5445d948a1ac412b559724fdbe [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:33.109191 18543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 091dac5445d948a1ac412b559724fdbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 091dac5445d948a1ac412b559724fdbe. Latest consensus state: current_term: 1 leader_uuid: "091dac5445d948a1ac412b559724fdbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091dac5445d948a1ac412b559724fdbe" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44937 } } }
I20260806 14:58:33.109273 18543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 091dac5445d948a1ac412b559724fdbe [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:33.109351 18549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 14:58:33.109859 18549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 14:58:33.111372 18549 catalog_manager.cc:1347] Generated new cluster ID: bd37292d45c047a6831fc8fc5d439818
I20260806 14:58:33.111428 18549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 14:58:33.116773 18549 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 14:58:33.117311 18549 catalog_manager.cc:1504] Loading token signing keys...
I20260806 14:58:33.121304 18549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 091dac5445d948a1ac412b559724fdbe: Generated new TSK 0
I20260806 14:58:33.121469 18549 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 14:58:33.143989 18578 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 14:58:33.144088 18578 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 14:58:33.144426 18538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:33.129920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52882 (local address 127.11.8.126:44937)
0806 14:58:33.129969 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:33.129971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:33.130079 (+ 108us) server_negotiation.cc:407] Connection header received
0806 14:58:33.130133 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:33.130134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:33.130148 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:33.130220 (+ 72us) server_negotiation.cc:226] Negotiated authn=JWT
0806 14:58:33.130354 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:33.131193 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:33.131953 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:33.132291 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:33.132359 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:33.132517 (+ 158us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 14:58:33.144201 (+ 11684us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45287/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 14:58:33.144351 (+ 150us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52882: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45287/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":27}
W20260806 14:58:33.144493 18538 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52882: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45287/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 14:58:33.144476 18576 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:33.129852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:44937 (local address 127.0.0.1:52882)
0806 14:58:33.129995 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:33.130005 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:33.130071 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:33.130223 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:33.130224 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:33.130226 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 14:58:33.130289 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:33.130291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:33.131288 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:33.131289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:33.131863 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:33.131864 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:33.132375 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:33.132376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:33.132417 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:33.132421 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 14:58:33.144312 (+ 11891us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45287/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 14:58:33.144404 (+ 92us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.8.126:44937: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45287/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":92,"threads_started":1}
I20260806 14:58:33.144795 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 18456
2026-08-06T14:58:33Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (236 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 14:58:33.191108 11297 mini_oidc.cc:152] Starting JWKS server
I20260806 14:58:33.191819 11297 webserver.cc:460] Webserver started at https://127.0.0.1:42685/ using document root <none> and password file <none>
I20260806 14:58:33.191891 11297 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 14:58:33.192056 11297 webserver.cc:460] Webserver started at http://127.0.0.1:35797/ using document root <none> and password file <none>
2026-08-06T14:58:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T14:58:33Z Disabled control of system clock
I20260806 14:58:33.200969 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:37365
--webserver_interface=127.11.8.126
--webserver_port=0
--builtin_ntp_servers=127.11.8.84:38981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:37365
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42685/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 14:58:33.282441 18585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:33.282647 18585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:33.282670 18585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:33.283764 18585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 14:58:33.283798 18585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:33.283811 18585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:33.283824 18585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 14:58:33.283836 18585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 14:58:33.283849 18585 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 14:58:33.283860 18585 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42685/jwks.json
I20260806 14:58:33.285225 18585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:38981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:37365
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:37365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42685/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:33.285419 18585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:33.285635 18585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:33.290364 18597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:33.290372 18595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:33.290395 18593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:33.290690 18585 server_base.cc:1034] running on GCE node
I20260806 14:58:33.290974 18585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:33.291258 18585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:33.292425 18585 hybrid_clock.cc:648] HybridClock initialized: now 1786028313292399 us; error 60 us; skew 500 ppm
I20260806 14:58:33.293650 18585 webserver.cc:460] Webserver started at http://127.11.8.126:46161/ using document root <none> and password file <none>
I20260806 14:58:33.293884 18585 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:33.293953 18585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:33.294094 18585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:33.295307 18585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "417dd99b65af477aac9e523ac496be23"
format_stamp: "Formatted at 2026-08-06 14:58:33 on dist-test-slave-68xk"
server_key: "14c40d00ee50e5af4cdfa7b7529ccf7f"
server_key_iv: "13add6aec1918310ebd82a47892bcd4d"
server_key_version: "encryptionkey@0"
I20260806 14:58:33.295722 18585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "417dd99b65af477aac9e523ac496be23"
format_stamp: "Formatted at 2026-08-06 14:58:33 on dist-test-slave-68xk"
server_key: "14c40d00ee50e5af4cdfa7b7529ccf7f"
server_key_iv: "13add6aec1918310ebd82a47892bcd4d"
server_key_version: "encryptionkey@0"
I20260806 14:58:33.297021 18585 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 14:58:33.297816 18605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:33.298003 18585 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 14:58:33.298144 18585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "417dd99b65af477aac9e523ac496be23"
format_stamp: "Formatted at 2026-08-06 14:58:33 on dist-test-slave-68xk"
server_key: "14c40d00ee50e5af4cdfa7b7529ccf7f"
server_key_iv: "13add6aec1918310ebd82a47892bcd4d"
server_key_version: "encryptionkey@0"
I20260806 14:58:33.298226 18585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:33.328662 18585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:33.328996 18585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:33.329162 18585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:33.333432 18585 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:37365
I20260806 14:58:33.333489 18672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:37365 every 8 connection(s)
I20260806 14:58:33.333865 18585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 14:58:33.334635 18673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:33.335651 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 18585
I20260806 14:58:33.335783 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 14:58:33.335974 11297 external_mini_cluster.cc:1428] Setting key 3eee272ac47acf8566f58d9d78b6e555
I20260806 14:58:33.338227 18673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 417dd99b65af477aac9e523ac496be23: Bootstrap starting.
I20260806 14:58:33.338966 18673 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 417dd99b65af477aac9e523ac496be23: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:33.339279 18673 log.cc:824] T 00000000000000000000000000000000 P 417dd99b65af477aac9e523ac496be23: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:33.340062 18673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 417dd99b65af477aac9e523ac496be23: No bootstrap required, opened a new log
I20260806 14:58:33.342262 18677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:33.336505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34400 (local address 127.11.8.126:37365)
0806 14:58:33.336730 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:33.336739 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:33.336774 (+ 35us) server_negotiation.cc:407] Connection header received
0806 14:58:33.337338 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:33.337348 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:33.337432 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:33.337587 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:33.337752 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:33.339087 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:33.340075 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:33.340675 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:33.340805 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:33.341499 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:33.341504 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:33.341504 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 14:58:33.341708 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:33.341752 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 14:58:33.341767 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:33.341836 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:33.341904 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 14:58:33.341992 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":86,"threads_started":1}
I20260806 14:58:33.342631 18673 raft_consensus.cc:348] T 00000000000000000000000000000000 P 417dd99b65af477aac9e523ac496be23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "417dd99b65af477aac9e523ac496be23" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 37365 } }
I20260806 14:58:33.342811 18673 raft_consensus.cc:374] T 00000000000000000000000000000000 P 417dd99b65af477aac9e523ac496be23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:33.342849 18673 raft_consensus.cc:729] T 00000000000000000000000000000000 P 417dd99b65af477aac9e523ac496be23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 417dd99b65af477aac9e523ac496be23, State: Initialized, Role: FOLLOWER
I20260806 14:58:33.342953 18673 consensus_queue.cc:260] T 00000000000000000000000000000000 P 417dd99b65af477aac9e523ac496be23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "417dd99b65af477aac9e523ac496be23" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 37365 } }
I20260806 14:58:33.343036 18673 raft_consensus.cc:388] T 00000000000000000000000000000000 P 417dd99b65af477aac9e523ac496be23 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 14:58:33.343070 18673 raft_consensus.cc:482] T 00000000000000000000000000000000 P 417dd99b65af477aac9e523ac496be23 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 14:58:33.343142 18673 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 417dd99b65af477aac9e523ac496be23 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:33.343458 11297 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 14:58:33.343796 18673 raft_consensus.cc:504] T 00000000000000000000000000000000 P 417dd99b65af477aac9e523ac496be23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "417dd99b65af477aac9e523ac496be23" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 37365 } }
I20260806 14:58:33.343915 18673 leader_election.cc:302] T 00000000000000000000000000000000 P 417dd99b65af477aac9e523ac496be23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 417dd99b65af477aac9e523ac496be23; no voters:
I20260806 14:58:33.344100 18673 leader_election.cc:288] T 00000000000000000000000000000000 P 417dd99b65af477aac9e523ac496be23 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 14:58:33.344115 18679 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 417dd99b65af477aac9e523ac496be23 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:33.344259 18679 raft_consensus.cc:686] T 00000000000000000000000000000000 P 417dd99b65af477aac9e523ac496be23 [term 1 LEADER]: Becoming Leader. State: Replica: 417dd99b65af477aac9e523ac496be23, State: Running, Role: LEADER
I20260806 14:58:33.344342 18679 consensus_queue.cc:237] T 00000000000000000000000000000000 P 417dd99b65af477aac9e523ac496be23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "417dd99b65af477aac9e523ac496be23" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 37365 } }
I20260806 14:58:33.344373 18673 sys_catalog.cc:564] T 00000000000000000000000000000000 P 417dd99b65af477aac9e523ac496be23 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 14:58:33.344660 18682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 417dd99b65af477aac9e523ac496be23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 417dd99b65af477aac9e523ac496be23. Latest consensus state: current_term: 1 leader_uuid: "417dd99b65af477aac9e523ac496be23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "417dd99b65af477aac9e523ac496be23" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 37365 } } }
I20260806 14:58:33.344660 18684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 417dd99b65af477aac9e523ac496be23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "417dd99b65af477aac9e523ac496be23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "417dd99b65af477aac9e523ac496be23" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 37365 } } }
I20260806 14:58:33.344833 18682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 417dd99b65af477aac9e523ac496be23 [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:33.344852 18684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 417dd99b65af477aac9e523ac496be23 [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:33.345353 18695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 14:58:33.345851 18695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 14:58:33.347304 18695 catalog_manager.cc:1347] Generated new cluster ID: b48cee46e9e04f6bb117b641f901c27a
I20260806 14:58:33.347364 18695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 14:58:33.360773 18695 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 14:58:33.361269 18695 catalog_manager.cc:1504] Loading token signing keys...
I20260806 14:58:33.369635 18695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 417dd99b65af477aac9e523ac496be23: Generated new TSK 0
I20260806 14:58:33.369849 18695 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 14:58:33.383653 18677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:33.380004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34414 (local address 127.11.8.126:37365)
0806 14:58:33.380069 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:33.380071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:33.380195 (+ 124us) server_negotiation.cc:407] Connection header received
0806 14:58:33.380267 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:33.380268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:33.380278 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:33.380344 (+ 66us) server_negotiation.cc:226] Negotiated authn=JWT
0806 14:58:33.380500 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:33.380988 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:33.382126 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:33.382322 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:33.382376 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:33.382594 (+ 218us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 14:58:33.383422 (+ 828us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42685/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 14:58:33.383577 (+ 155us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34414: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42685/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":37}
W20260806 14:58:33.383800 18677 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34414: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42685/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 14:58:33.384948 18714 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:33.379936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:37365 (local address 127.0.0.1:34414)
0806 14:58:33.380116 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:33.380127 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:33.380197 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:33.380362 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:33.380363 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:33.380364 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0806 14:58:33.380435 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:33.380437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:33.381070 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:33.381071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:33.382040 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:33.382043 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:33.382404 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:33.382405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:33.382490 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:33.382497 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 14:58:33.383530 (+ 1033us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42685/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 14:58:33.383623 (+ 93us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.8.126:37365: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42685/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":113,"threads_started":1}
I20260806 14:58:33.385134 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 18585
E20260806 14:58:33.399861 18715 webserver.cc:533] Webserver: sslize failed:
E20260806 14:58:33.399930 18715 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T14:58:33Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (238 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T14:58:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T14:58:33Z Disabled control of system clock
I20260806 14:58:33.438452 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:35743
--webserver_interface=127.11.8.126
--webserver_port=0
--builtin_ntp_servers=127.11.8.84:46311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:35743
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028287744255-11297-0/testchainca.cert with env {}
W20260806 14:58:33.519161 18719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:33.519371 18719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:33.519392 18719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:33.521000 18719 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 14:58:33.521056 18719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:33.521080 18719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:33.521104 18719 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 14:58:33.521126 18719 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 14:58:33.521153 18719 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 14:58:33.521171 18719 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 14:58:33.523160 18719 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:46311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:35743
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:35743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028287744255-11297-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:33.523461 18719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:33.523679 18719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:33.529146 18727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:33.529150 18726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:33.529675 18719 server_base.cc:1034] running on GCE node
W20260806 14:58:33.530031 18730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:33.530320 18719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:33.530709 18719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:33.531905 18719 hybrid_clock.cc:648] HybridClock initialized: now 1786028313531895 us; error 38 us; skew 500 ppm
I20260806 14:58:33.533483 18719 webserver.cc:460] Webserver started at http://127.11.8.126:41757/ using document root <none> and password file <none>
I20260806 14:58:33.533792 18719 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:33.533874 18719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:33.534019 18719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:33.535578 18719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "9a0ba3e935bb48f98cf5a6478696e09a"
format_stamp: "Formatted at 2026-08-06 14:58:33 on dist-test-slave-68xk"
server_key: "ccc8e85939dcc82198567f04600ffe1f"
server_key_iv: "d9f953e04dcb93f67ef22b8ff338428f"
server_key_version: "encryptionkey@0"
I20260806 14:58:33.536039 18719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "9a0ba3e935bb48f98cf5a6478696e09a"
format_stamp: "Formatted at 2026-08-06 14:58:33 on dist-test-slave-68xk"
server_key: "ccc8e85939dcc82198567f04600ffe1f"
server_key_iv: "d9f953e04dcb93f67ef22b8ff338428f"
server_key_version: "encryptionkey@0"
I20260806 14:58:33.537966 18719 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20260806 14:58:33.539026 18739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:33.539227 18719 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 14:58:33.539330 18719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "9a0ba3e935bb48f98cf5a6478696e09a"
format_stamp: "Formatted at 2026-08-06 14:58:33 on dist-test-slave-68xk"
server_key: "ccc8e85939dcc82198567f04600ffe1f"
server_key_iv: "d9f953e04dcb93f67ef22b8ff338428f"
server_key_version: "encryptionkey@0"
I20260806 14:58:33.539408 18719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:33.549160 18719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:33.549394 18719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:33.549494 18719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:33.553426 18719 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:35743
I20260806 14:58:33.553479 18811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:35743 every 8 connection(s)
I20260806 14:58:33.553809 18719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 14:58:33.554842 18812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:33.557150 18812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a0ba3e935bb48f98cf5a6478696e09a: Bootstrap starting.
I20260806 14:58:33.557678 18812 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a0ba3e935bb48f98cf5a6478696e09a: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:33.557914 18812 log.cc:824] T 00000000000000000000000000000000 P 9a0ba3e935bb48f98cf5a6478696e09a: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:33.558629 18812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a0ba3e935bb48f98cf5a6478696e09a: No bootstrap required, opened a new log
I20260806 14:58:33.560266 18812 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a0ba3e935bb48f98cf5a6478696e09a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a0ba3e935bb48f98cf5a6478696e09a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35743 } }
I20260806 14:58:33.560405 18812 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a0ba3e935bb48f98cf5a6478696e09a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:33.560426 18812 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a0ba3e935bb48f98cf5a6478696e09a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a0ba3e935bb48f98cf5a6478696e09a, State: Initialized, Role: FOLLOWER
I20260806 14:58:33.560524 18812 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a0ba3e935bb48f98cf5a6478696e09a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a0ba3e935bb48f98cf5a6478696e09a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35743 } }
I20260806 14:58:33.560572 18812 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a0ba3e935bb48f98cf5a6478696e09a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 14:58:33.560595 18812 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a0ba3e935bb48f98cf5a6478696e09a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 14:58:33.560626 18812 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a0ba3e935bb48f98cf5a6478696e09a [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:33.561092 18812 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a0ba3e935bb48f98cf5a6478696e09a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a0ba3e935bb48f98cf5a6478696e09a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35743 } }
I20260806 14:58:33.561175 18812 leader_election.cc:302] T 00000000000000000000000000000000 P 9a0ba3e935bb48f98cf5a6478696e09a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a0ba3e935bb48f98cf5a6478696e09a; no voters:
I20260806 14:58:33.561333 18812 leader_election.cc:288] T 00000000000000000000000000000000 P 9a0ba3e935bb48f98cf5a6478696e09a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 14:58:33.561365 18815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a0ba3e935bb48f98cf5a6478696e09a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:33.561480 18815 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a0ba3e935bb48f98cf5a6478696e09a [term 1 LEADER]: Becoming Leader. State: Replica: 9a0ba3e935bb48f98cf5a6478696e09a, State: Running, Role: LEADER
I20260806 14:58:33.561594 18812 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a0ba3e935bb48f98cf5a6478696e09a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 14:58:33.561569 18815 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a0ba3e935bb48f98cf5a6478696e09a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a0ba3e935bb48f98cf5a6478696e09a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35743 } }
I20260806 14:58:33.561959 18818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a0ba3e935bb48f98cf5a6478696e09a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a0ba3e935bb48f98cf5a6478696e09a. Latest consensus state: current_term: 1 leader_uuid: "9a0ba3e935bb48f98cf5a6478696e09a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a0ba3e935bb48f98cf5a6478696e09a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35743 } } }
I20260806 14:58:33.562052 18818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a0ba3e935bb48f98cf5a6478696e09a [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:33.562098 18816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a0ba3e935bb48f98cf5a6478696e09a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a0ba3e935bb48f98cf5a6478696e09a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a0ba3e935bb48f98cf5a6478696e09a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35743 } } }
I20260806 14:58:33.562196 18816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a0ba3e935bb48f98cf5a6478696e09a [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:33.562294 18822 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 14:58:33.562815 18822 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 14:58:33.562949 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 18719
I20260806 14:58:33.563068 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 14:58:33.563299 11297 external_mini_cluster.cc:1428] Setting key e6e2c27313f6e20bb27c552e4a25d435
I20260806 14:58:33.564751 18822 catalog_manager.cc:1347] Generated new cluster ID: a8f0724bff1c424f88a96b0e9c0c16c0
I20260806 14:58:33.564812 18822 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 14:58:33.569705 18840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:33.563776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42088 (local address 127.11.8.126:35743)
0806 14:58:33.563982 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:33.563990 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:33.564171 (+ 181us) server_negotiation.cc:407] Connection header received
0806 14:58:33.564342 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:33.564350 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:33.564425 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:33.564567 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:33.564807 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:33.566263 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:33.567435 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:33.568023 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:33.568130 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:33.568887 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:33.568890 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:33.568891 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 14:58:33.569093 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:33.569132 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 14:58:33.569145 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:33.569210 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:33.569317 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 14:58:33.569387 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":79,"threads_started":1}
I20260806 14:58:33.570633 11297 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 14:58:33.584424 18822 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 14:58:33.584893 18822 catalog_manager.cc:1504] Loading token signing keys...
I20260806 14:58:33.587327 18822 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a0ba3e935bb48f98cf5a6478696e09a: Generated new TSK 0
I20260806 14:58:33.587504 18822 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 14:58:34.040115 18840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:33.592248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42102 (local address 127.11.8.126:35743)
0806 14:58:33.592314 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:33.592316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:33.592424 (+ 108us) server_negotiation.cc:407] Connection header received
0806 14:58:33.592502 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:33.592503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:33.592513 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:33.592572 (+ 59us) server_negotiation.cc:226] Negotiated authn=JWT
0806 14:58:33.592726 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:33.593226 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:33.594516 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:33.594736 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:33.594788 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:33.594988 (+ 200us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 14:58:34.039881 (+444893us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 14:58:34.040054 (+ 173us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42102: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 14:58:34.040195 18840 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42102: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 14:58:34.040313 18848 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:33.592190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:35743 (local address 127.0.0.1:42102)
0806 14:58:33.592347 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:33.592360 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:33.592432 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:33.592595 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:33.592597 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:33.592599 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 14:58:33.592661 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:33.592664 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:33.593310 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:33.593311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:33.594366 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:33.594369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:33.594819 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:33.594821 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:33.594890 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:33.594896 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 14:58:34.040029 (+445133us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 14:58:34.040157 (+ 128us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.11.8.126:35743: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":100,"threads_started":1}
I20260806 14:58:34.040705 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 18719
2026-08-06T14:58:34Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (634 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (92 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (92 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 14:58:34 dist-test-slave-68xk krb5kdc[18861](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 14:58:34 dist-test-slave-68xk krb5kdc[18861](info): set up 2 sockets
Aug 06 14:58:34 dist-test-slave-68xk krb5kdc[18861](info): commencing operation
krb5kdc: starting...
W20260806 14:58:35.430107 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 14:58:35 dist-test-slave-68xk krb5kdc[18861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T14:58:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T14:58:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 14:58:35.553720 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:35135
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:37977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:35135
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:35.638245 18877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:35.638592 18877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:35.638634 18877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:35.639776 18877 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 14:58:35.639842 18877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:35.639866 18877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:35.639894 18877 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 14:58:35.639923 18877 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 14:58:35.641343 18877 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:37977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:35135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:35135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:35.641618 18877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:35.641860 18877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:35.646721 18890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:35.646934 18877 server_base.cc:1034] running on GCE node
W20260806 14:58:35.647013 18892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:35.646965 18889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:35.647321 18877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:35.647653 18877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:35.648810 18877 hybrid_clock.cc:648] HybridClock initialized: now 1786028315648795 us; error 29 us; skew 500 ppm
Aug 06 14:58:35 dist-test-slave-68xk krb5kdc[18861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028315, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:35.656699 18877 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 14:58:35.657141 18877 webserver.cc:460] Webserver started at http://127.11.8.126:36257/ using document root <none> and password file <none>
I20260806 14:58:35.657393 18877 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:35.657465 18877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:35.657614 18877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:35.658746 18877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "8a65dd11dac845b588242bc4bad689f7"
format_stamp: "Formatted at 2026-08-06 14:58:35 on dist-test-slave-68xk"
server_key: "7ac84e0f8a24ddf81ca9afa8db8ce0ab"
server_key_iv: "bacd45967b4a087c6f3c9ba39cf334e2"
server_key_version: "encryptionkey@0"
I20260806 14:58:35.659111 18877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "8a65dd11dac845b588242bc4bad689f7"
format_stamp: "Formatted at 2026-08-06 14:58:35 on dist-test-slave-68xk"
server_key: "7ac84e0f8a24ddf81ca9afa8db8ce0ab"
server_key_iv: "bacd45967b4a087c6f3c9ba39cf334e2"
server_key_version: "encryptionkey@0"
I20260806 14:58:35.660465 18877 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 14:58:35.661187 18899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:35.661489 18877 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:35.661600 18877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "8a65dd11dac845b588242bc4bad689f7"
format_stamp: "Formatted at 2026-08-06 14:58:35 on dist-test-slave-68xk"
server_key: "7ac84e0f8a24ddf81ca9afa8db8ce0ab"
server_key_iv: "bacd45967b4a087c6f3c9ba39cf334e2"
server_key_version: "encryptionkey@0"
I20260806 14:58:35.661675 18877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:35.677284 18877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:35.679790 18877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:35.679961 18877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:35.684252 18877 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:35135
I20260806 14:58:35.684300 18965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:35135 every 8 connection(s)
I20260806 14:58:35.684646 18877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 14:58:35.685534 18966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:35.687947 18966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a65dd11dac845b588242bc4bad689f7: Bootstrap starting.
I20260806 14:58:35.688609 18966 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a65dd11dac845b588242bc4bad689f7: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:35.688915 18966 log.cc:824] T 00000000000000000000000000000000 P 8a65dd11dac845b588242bc4bad689f7: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:35.689177 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 18877
I20260806 14:58:35.689281 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 14:58:35.689486 11297 external_mini_cluster.cc:1428] Setting key 50e26425a00ef7d236838582f1a6ca81
I20260806 14:58:35.689759 18966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a65dd11dac845b588242bc4bad689f7: No bootstrap required, opened a new log
I20260806 14:58:35.691730 18966 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a65dd11dac845b588242bc4bad689f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a65dd11dac845b588242bc4bad689f7" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35135 } }
I20260806 14:58:35.691898 18966 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a65dd11dac845b588242bc4bad689f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:35.691967 18966 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a65dd11dac845b588242bc4bad689f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a65dd11dac845b588242bc4bad689f7, State: Initialized, Role: FOLLOWER
I20260806 14:58:35.692108 18966 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a65dd11dac845b588242bc4bad689f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a65dd11dac845b588242bc4bad689f7" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35135 } }
I20260806 14:58:35.692206 18966 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a65dd11dac845b588242bc4bad689f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 14:58:35.692252 18966 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a65dd11dac845b588242bc4bad689f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 14:58:35.692302 18966 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a65dd11dac845b588242bc4bad689f7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:35.693184 18966 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a65dd11dac845b588242bc4bad689f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a65dd11dac845b588242bc4bad689f7" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35135 } }
I20260806 14:58:35.693324 18966 leader_election.cc:302] T 00000000000000000000000000000000 P 8a65dd11dac845b588242bc4bad689f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a65dd11dac845b588242bc4bad689f7; no voters:
I20260806 14:58:35.693524 18966 leader_election.cc:288] T 00000000000000000000000000000000 P 8a65dd11dac845b588242bc4bad689f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 14:58:35.693550 18972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a65dd11dac845b588242bc4bad689f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:35.693676 18972 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a65dd11dac845b588242bc4bad689f7 [term 1 LEADER]: Becoming Leader. State: Replica: 8a65dd11dac845b588242bc4bad689f7, State: Running, Role: LEADER
I20260806 14:58:35.693813 18966 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a65dd11dac845b588242bc4bad689f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 14:58:35.693821 18972 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a65dd11dac845b588242bc4bad689f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a65dd11dac845b588242bc4bad689f7" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35135 } }
I20260806 14:58:35.694283 18975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a65dd11dac845b588242bc4bad689f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a65dd11dac845b588242bc4bad689f7. Latest consensus state: current_term: 1 leader_uuid: "8a65dd11dac845b588242bc4bad689f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a65dd11dac845b588242bc4bad689f7" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35135 } } }
I20260806 14:58:35.694376 18975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a65dd11dac845b588242bc4bad689f7 [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:35.694621 18973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a65dd11dac845b588242bc4bad689f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a65dd11dac845b588242bc4bad689f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a65dd11dac845b588242bc4bad689f7" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35135 } } }
I20260806 14:58:35.694907 18973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a65dd11dac845b588242bc4bad689f7 [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:35.695149 18981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 14:58:35.695734 18981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 14:58:35.697456 18981 catalog_manager.cc:1347] Generated new cluster ID: 4de54a7dfdc74d10ad2f46dc09c7b9e3
I20260806 14:58:35.697512 18981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 14:58:35 dist-test-slave-68xk krb5kdc[18861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:35.701864 18971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:35.690089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34086 (local address 127.11.8.126:35135)
0806 14:58:35.690315 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:35.690332 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:35.690358 (+ 26us) server_negotiation.cc:407] Connection header received
0806 14:58:35.691715 (+ 1357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:35.691723 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:35.691793 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:35.691953 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:35.692726 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:35.693865 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:35.694982 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:35.695652 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:35.695777 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:35.698462 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:35.698466 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:35.698466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:35.698479 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:35.700264 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:35.700643 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:35.700646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:35.700647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:35.700757 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:35.700969 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:35.700970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:35.700970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:35.701212 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:35.701410 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:35.701477 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 14:58:35.701715 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":118,"threads_started":1}
I20260806 14:58:35.728066 18981 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 14:58:35.728828 18981 catalog_manager.cc:1504] Loading token signing keys...
I20260806 14:58:35.738853 18981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a65dd11dac845b588242bc4bad689f7: Generated new TSK 0
I20260806 14:58:35.739060 18981 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
I20260806 14:58:35.770053 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:0
--local_ip_for_outbound_sockets=127.11.8.65
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:35135
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:37977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:35.852835 18999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:35.853037 18999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:35.853061 18999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:35.854213 18999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:35.854259 18999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:35.854313 18999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 14:58:35.855885 18999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:37977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:35135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:35.856120 18999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:35.856369 18999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:35.861414 19004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:35.861451 19005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:35.861515 19008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:35.861682 18999 server_base.cc:1034] running on GCE node
I20260806 14:58:35.861965 18999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:35.862260 18999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:35.863430 18999 hybrid_clock.cc:648] HybridClock initialized: now 1786028315863408 us; error 36 us; skew 500 ppm
Aug 06 14:58:35 dist-test-slave-68xk krb5kdc[18861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028315, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:35.866093 18999 init.cc:377] Logged in from keytab as kudu/127.11.8.65@KRBTEST.COM (short username kudu)
I20260806 14:58:35.866703 18999 webserver.cc:460] Webserver started at http://127.11.8.65:45667/ using document root <none> and password file <none>
I20260806 14:58:35.866962 18999 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:35.867039 18999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:35.867210 18999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:35.868341 18999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-0/data/instance:
uuid: "fd16b5c098874853ae090be0a531d2e9"
format_stamp: "Formatted at 2026-08-06 14:58:35 on dist-test-slave-68xk"
server_key: "daba75b499ccaef9086b862ae21dd000"
server_key_iv: "5ed3bd93bc482d755e27439a691c5cbf"
server_key_version: "encryptionkey@0"
I20260806 14:58:35.868738 18999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance:
uuid: "fd16b5c098874853ae090be0a531d2e9"
format_stamp: "Formatted at 2026-08-06 14:58:35 on dist-test-slave-68xk"
server_key: "daba75b499ccaef9086b862ae21dd000"
server_key_iv: "5ed3bd93bc482d755e27439a691c5cbf"
server_key_version: "encryptionkey@0"
I20260806 14:58:35.870263 18999 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 14:58:35.871155 19016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:35.871338 18999 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 14:58:35.871455 18999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "fd16b5c098874853ae090be0a531d2e9"
format_stamp: "Formatted at 2026-08-06 14:58:35 on dist-test-slave-68xk"
server_key: "daba75b499ccaef9086b862ae21dd000"
server_key_iv: "5ed3bd93bc482d755e27439a691c5cbf"
server_key_version: "encryptionkey@0"
I20260806 14:58:35.871552 18999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:35.882824 18999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:35.885139 18999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:35.885335 18999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:35.885629 18999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:35.885986 18999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:35.886051 18999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:35.886091 18999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:35.886137 18999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:35.893008 18999 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:42927
I20260806 14:58:35.893047 19170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:42927 every 8 connection(s)
I20260806 14:58:35.893407 18999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
I20260806 14:58:35.895650 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 18999
I20260806 14:58:35.895776 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance
I20260806 14:58:35.896005 11297 external_mini_cluster.cc:1428] Setting key f0905f9eb3e684d32241ac00c837fa2a
Aug 06 14:58:35 dist-test-slave-68xk krb5kdc[18861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028315, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:35.904845 18971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:35.894226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:53937 (local address 127.11.8.126:35135)
0806 14:58:35.894302 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:35.894304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:35.894629 (+ 325us) server_negotiation.cc:407] Connection header received
0806 14:58:35.895986 (+ 1357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:35.895987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:35.896005 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:35.896121 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:35.896975 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:35.897539 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:35.898728 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:35.898980 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:35.899094 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:35.901446 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:35.901448 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:35.901448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:35.901458 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:35.903321 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:35.903758 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:35.903759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:35.903760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:35.903829 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:35.904093 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:35.904094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:35.904094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:35.904253 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:35.904328 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:35.904627 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 14:58:35.904717 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 14:58:35.904882 19173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:35.894218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:35135 (local address 127.11.8.65:53937)
0806 14:58:35.894410 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:35.894528 (+ 118us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:35.895849 (+ 1321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:35.896206 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:35.896211 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:35.896615 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:35.896864 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:35.896870 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:35.897657 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:35.897658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:35.898630 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:35.898632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:35.899108 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:35.899109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:35.899248 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:35.899738 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:35.899747 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:35.901342 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:35.903415 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:35.903422 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:35.903424 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:35.903671 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:35.903906 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:35.903906 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:35.903907 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:35.904012 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:35.904329 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:35.904335 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:35.904469 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 14:58:35.904568 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 14:58:35.904705 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":91,"threads_started":1}
WARNING: no policy specified for kudu/127.11.8.66@KRBTEST.COM; defaulting to no policy
I20260806 14:58:35.905691 19171 heartbeater.cc:347] Connected to a master server at 127.11.8.126:35135
I20260806 14:58:35.905814 19171 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:35.906010 19171 heartbeater.cc:511] Master 127.11.8.126:35135 requested a full tablet report, sending...
I20260806 14:58:35.906666 18922 ts_manager.cc:194] Registered new tserver with Master: fd16b5c098874853ae090be0a531d2e9 (127.11.8.65:42927)
I20260806 14:58:35.907291 18922 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.65@KRBTEST.COM'} at 127.11.8.65:53937
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
I20260806 14:58:35.925230 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:0
--local_ip_for_outbound_sockets=127.11.8.66
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:35135
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:37977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:36.008126 19176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:36.008330 19176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:36.008353 19176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:36.009430 19176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:36.009464 19176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:36.009490 19176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 14:58:36.011029 19176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:37977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:35135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:36.011226 19176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:36.011399 19176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:36.016578 19182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:36.016568 19186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:36.016650 19183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:36.016819 19176 server_base.cc:1034] running on GCE node
I20260806 14:58:36.017019 19176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:36.017273 19176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:36.018481 19176 hybrid_clock.cc:648] HybridClock initialized: now 1786028316018472 us; error 39 us; skew 500 ppm
Aug 06 14:58:36 dist-test-slave-68xk krb5kdc[18861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028316, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:36.021098 19176 init.cc:377] Logged in from keytab as kudu/127.11.8.66@KRBTEST.COM (short username kudu)
I20260806 14:58:36.021530 19176 webserver.cc:460] Webserver started at http://127.11.8.66:33925/ using document root <none> and password file <none>
I20260806 14:58:36.021768 19176 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:36.021840 19176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:36.021991 19176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:36.023123 19176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-1/data/instance:
uuid: "12bb527263084e3c84fabb285086ebd9"
format_stamp: "Formatted at 2026-08-06 14:58:36 on dist-test-slave-68xk"
server_key: "be881af4cc10694785be56c0293243b0"
server_key_iv: "64d18c2670c6dd9272ee5de5cc957c95"
server_key_version: "encryptionkey@0"
I20260806 14:58:36.023491 19176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance:
uuid: "12bb527263084e3c84fabb285086ebd9"
format_stamp: "Formatted at 2026-08-06 14:58:36 on dist-test-slave-68xk"
server_key: "be881af4cc10694785be56c0293243b0"
server_key_iv: "64d18c2670c6dd9272ee5de5cc957c95"
server_key_version: "encryptionkey@0"
I20260806 14:58:36.024968 19176 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 14:58:36.025791 19197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:36.025992 19176 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:36.026103 19176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "12bb527263084e3c84fabb285086ebd9"
format_stamp: "Formatted at 2026-08-06 14:58:36 on dist-test-slave-68xk"
server_key: "be881af4cc10694785be56c0293243b0"
server_key_iv: "64d18c2670c6dd9272ee5de5cc957c95"
server_key_version: "encryptionkey@0"
I20260806 14:58:36.026178 19176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:36.043141 19176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:36.045115 19176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:36.045282 19176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:36.045522 19176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:36.045848 19176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:36.045905 19176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:36.045945 19176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:36.045974 19176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:36.052276 19176 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:36869
I20260806 14:58:36.052330 19347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:36869 every 8 connection(s)
I20260806 14:58:36.052631 19176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
Aug 06 14:58:36 dist-test-slave-68xk krb5kdc[18861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028316, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:36.060801 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 19176
I20260806 14:58:36.060916 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance
I20260806 14:58:36.061120 11297 external_mini_cluster.cc:1428] Setting key 94a230dee63a436daf947cea0318699a
I20260806 14:58:36.062458 18971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:36.053365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:33407 (local address 127.11.8.126:35135)
0806 14:58:36.053434 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:36.053435 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:36.053631 (+ 196us) server_negotiation.cc:407] Connection header received
0806 14:58:36.054630 (+ 999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:36.054630 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:36.054641 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:36.054711 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:36.055347 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:36.055881 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:36.056975 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:36.057154 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:36.057213 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:36.059463 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:36.059464 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:36.059465 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:36.059471 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:36.060846 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:36.061323 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:36.061324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:36.061324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:36.061386 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:36.061668 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:36.061670 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:36.061670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:36.061815 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:36.061893 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:36.062267 (+ 374us) server_negotiation.cc:299] Negotiation successful
0806 14:58:36.062345 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 14:58:36.062734 19351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:36.053312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:35135 (local address 127.11.8.66:33407)
0806 14:58:36.053503 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:36.053545 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:36.054534 (+ 989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:36.054788 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:36.054792 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:36.055082 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:36.055270 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:36.055274 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:36.055987 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:36.055988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:36.056894 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:36.056896 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:36.057238 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:36.057240 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:36.057353 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:36.057841 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:36.057852 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:36.059371 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:36.060939 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:36.060946 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:36.060949 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:36.061245 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:36.061459 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:36.061460 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:36.061461 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:36.061589 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:36.061890 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:36.061914 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:36.062081 (+ 167us) client_negotiation.cc:769] Sending connection context
0806 14:58:36.062245 (+ 164us) client_negotiation.cc:240] Negotiation successful
0806 14:58:36.062450 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":116,"threads_started":1}
I20260806 14:58:36.063256 19349 heartbeater.cc:347] Connected to a master server at 127.11.8.126:35135
I20260806 14:58:36.063380 19349 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:36.063556 19349 heartbeater.cc:511] Master 127.11.8.126:35135 requested a full tablet report, sending...
I20260806 14:58:36.063953 18922 ts_manager.cc:194] Registered new tserver with Master: 12bb527263084e3c84fabb285086ebd9 (127.11.8.66:36869)
I20260806 14:58:36.064338 18922 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.66@KRBTEST.COM'} at 127.11.8.66:33407
WARNING: no policy specified for kudu/127.11.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
I20260806 14:58:36.090237 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:0
--local_ip_for_outbound_sockets=127.11.8.67
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:35135
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:37977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:36.172320 19354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:36.172554 19354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:36.172595 19354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:36.173733 19354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:36.173807 19354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:36.173852 19354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 14:58:36.175441 19354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:37977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:35135
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:36.175709 19354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:36.175953 19354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:36.180949 19360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:36.181202 19362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:36.181218 19359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:36.181521 19354 server_base.cc:1034] running on GCE node
I20260806 14:58:36.181787 19354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:36.182030 19354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:36.183199 19354 hybrid_clock.cc:648] HybridClock initialized: now 1786028316183180 us; error 32 us; skew 500 ppm
Aug 06 14:58:36 dist-test-slave-68xk krb5kdc[18861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028316, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:36.186002 19354 init.cc:377] Logged in from keytab as kudu/127.11.8.67@KRBTEST.COM (short username kudu)
I20260806 14:58:36.186512 19354 webserver.cc:460] Webserver started at http://127.11.8.67:36779/ using document root <none> and password file <none>
I20260806 14:58:36.186748 19354 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:36.186818 19354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:36.186976 19354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:36.188020 19354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-2/data/instance:
uuid: "4a26d0f069c646b590588a0cf51d80f0"
format_stamp: "Formatted at 2026-08-06 14:58:36 on dist-test-slave-68xk"
server_key: "6d11293956ad68bb5820fabfcee19cc9"
server_key_iv: "9aa57ae740bd0ad1b2191689409e9c6a"
server_key_version: "encryptionkey@0"
I20260806 14:58:36.188386 19354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance:
uuid: "4a26d0f069c646b590588a0cf51d80f0"
format_stamp: "Formatted at 2026-08-06 14:58:36 on dist-test-slave-68xk"
server_key: "6d11293956ad68bb5820fabfcee19cc9"
server_key_iv: "9aa57ae740bd0ad1b2191689409e9c6a"
server_key_version: "encryptionkey@0"
I20260806 14:58:36.189731 19354 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 14:58:36.190526 19374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:36.190687 19354 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 14:58:36.190793 19354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "4a26d0f069c646b590588a0cf51d80f0"
format_stamp: "Formatted at 2026-08-06 14:58:36 on dist-test-slave-68xk"
server_key: "6d11293956ad68bb5820fabfcee19cc9"
server_key_iv: "9aa57ae740bd0ad1b2191689409e9c6a"
server_key_version: "encryptionkey@0"
I20260806 14:58:36.190883 19354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:36.209342 19354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:36.211943 19354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:36.212150 19354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:36.212417 19354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:36.212755 19354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:36.212833 19354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:36.212894 19354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:36.212925 19354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:36.218930 19354 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:40465
I20260806 14:58:36.219049 19538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:40465 every 8 connection(s)
I20260806 14:58:36.219338 19354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
I20260806 14:58:36.225541 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 19354
I20260806 14:58:36.225659 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance
I20260806 14:58:36.225847 11297 external_mini_cluster.cc:1428] Setting key 473b03137c874291720ad095e4cbb6e3
Aug 06 14:58:36 dist-test-slave-68xk krb5kdc[18861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028316, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:36.229255 18971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:36.220191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:46519 (local address 127.11.8.126:35135)
0806 14:58:36.220261 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:36.220263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:36.220474 (+ 211us) server_negotiation.cc:407] Connection header received
0806 14:58:36.221472 (+ 998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:36.221473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:36.221483 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:36.221545 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:36.222170 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:36.222747 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:36.223830 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:36.224012 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:36.224067 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:36.226462 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:36.226463 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:36.226463 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:36.226471 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:36.227944 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:36.228366 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:36.228367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:36.228367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:36.228419 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:36.228642 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:36.228642 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:36.228642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:36.228741 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:36.228791 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:36.229113 (+ 322us) server_negotiation.cc:299] Negotiation successful
0806 14:58:36.229176 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 14:58:36.229452 19542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:36.220133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:35135 (local address 127.11.8.67:46519)
0806 14:58:36.220322 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:36.220369 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:36.221372 (+ 1003us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:36.221613 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:36.221616 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:36.221887 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:36.222087 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:36.222092 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:36.222836 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:36.222837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:36.223747 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:36.223749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:36.224094 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:36.224095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:36.224197 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:36.224638 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:36.224647 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:36.226353 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:36.228021 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:36.228028 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:36.228030 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:36.228296 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:36.228481 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:36.228482 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:36.228483 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:36.228585 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:36.228798 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:36.228803 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:36.228944 (+ 141us) client_negotiation.cc:769] Sending connection context
0806 14:58:36.229082 (+ 138us) client_negotiation.cc:240] Negotiation successful
0806 14:58:36.229231 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":104,"threads_started":1}
I20260806 14:58:36.229940 19540 heartbeater.cc:347] Connected to a master server at 127.11.8.126:35135
I20260806 14:58:36.230052 19540 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:36.230247 19540 heartbeater.cc:511] Master 127.11.8.126:35135 requested a full tablet report, sending...
I20260806 14:58:36.230682 18922 ts_manager.cc:194] Registered new tserver with Master: 4a26d0f069c646b590588a0cf51d80f0 (127.11.8.67:40465)
I20260806 14:58:36.231098 18922 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.67@KRBTEST.COM'} at 127.11.8.67:46519
I20260806 14:58:36.237793 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/test-admin.keytab.
Aug 06 14:58:36 dist-test-slave-68xk krb5kdc[18861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:36.255374 11297 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 14:58:36.257301 19553 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:36.256395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34294 (local address 127.11.8.65:42927)
0806 14:58:36.256602 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:36.256607 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:36.256616 (+ 9us) server_negotiation.cc:407] Connection header received
0806 14:58:36.256643 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:36.256645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:36.256660 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:36.256765 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:36.257205 (+ 440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34294: BlockingRecv error: recv got EOF from 127.0.0.1:34294 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":88,"threads_started":1}
W20260806 14:58:36.257400 11297 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/krb5cc' of type FILE
Aug 06 14:58:36 dist-test-slave-68xk krb5kdc[18861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:36.258725 11297 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 14:58:36 dist-test-slave-68xk krb5kdc[18861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:36.259706 11297 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 14:58:36 dist-test-slave-68xk krb5kdc[18861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.65@KRBTEST.COM
I20260806 14:58:36.268599 19553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:36.260317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34296 (local address 127.11.8.65:42927)
0806 14:58:36.260378 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:36.260379 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:36.260425 (+ 46us) server_negotiation.cc:407] Connection header received
0806 14:58:36.260485 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:36.260486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:36.260496 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:36.260548 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:36.260983 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:36.261706 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:36.262532 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:36.262869 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:36.262927 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:36.265376 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:36.265425 (+ 49us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:36.265425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:36.265435 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:36.267407 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:36.267771 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:36.267774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:36.267774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:36.267831 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:36.268033 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:36.268035 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:36.268035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:36.268181 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:36.268299 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:36.268368 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 14:58:36.268436 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 14:58:36.270764 19553 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:36.270168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34308 (local address 127.11.8.65:42927)
0806 14:58:36.270230 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:36.270231 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:36.270247 (+ 16us) server_negotiation.cc:407] Connection header received
0806 14:58:36.270299 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:36.270300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:36.270309 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:36.270355 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:36.270726 (+ 371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34308: BlockingRecv error: recv got EOF from 127.0.0.1:34308 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 14:58:36.270910 11297 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/krb5cc)
Aug 06 14:58:36 dist-test-slave-68xk krb5kdc[18861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:36.271940 11297 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 14:58:36 dist-test-slave-68xk krb5kdc[18861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:36.272791 11297 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 14:58:36 dist-test-slave-68xk krb5kdc[18861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.65@KRBTEST.COM
I20260806 14:58:36.280223 19553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:36.273351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34312 (local address 127.11.8.65:42927)
0806 14:58:36.273406 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:36.273408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:36.273462 (+ 54us) server_negotiation.cc:407] Connection header received
0806 14:58:36.273526 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:36.273527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:36.273537 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:36.273590 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:36.273984 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:36.274542 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:36.275307 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:36.275523 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:36.275608 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:36.277594 (+ 1986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:36.277595 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:36.277596 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:36.277603 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:36.279021 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:36.279430 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:36.279432 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:36.279432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:36.279499 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:36.279719 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:36.279720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:36.279721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:36.279825 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:36.279935 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:36.280034 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 14:58:36.280122 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260806 14:58:36.281857 11297 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786028287744255-11297-0/krb5kdc/krb5cc' of type FILE
W20260806 14:58:36.281845 19553 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:36.281257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34318 (local address 127.11.8.65:42927)
0806 14:58:36.281332 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:36.281334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:36.281343 (+ 9us) server_negotiation.cc:407] Connection header received
0806 14:58:36.281413 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:36.281414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:36.281422 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:36.281473 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:36.281779 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34318: BlockingRecv error: recv got EOF from 127.0.0.1:34318 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 14:58:36 dist-test-slave-68xk krb5kdc[18861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:36.282938 11297 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 14:58:36 dist-test-slave-68xk krb5kdc[18861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:36.283818 11297 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 14:58:36 dist-test-slave-68xk krb5kdc[18861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.65@KRBTEST.COM
I20260806 14:58:36.291105 19553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:36.284427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34330 (local address 127.11.8.65:42927)
0806 14:58:36.284494 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:36.284496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:36.284540 (+ 44us) server_negotiation.cc:407] Connection header received
0806 14:58:36.284621 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:36.284621 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:36.284630 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:36.284675 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:36.285189 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:36.285656 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:36.286359 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:36.286640 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:36.286717 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:36.288681 (+ 1964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:36.288683 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:36.288683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:36.288690 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:36.289985 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:36.290327 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:36.290329 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:36.290329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:36.290389 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:36.290619 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:36.290620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:36.290620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:36.290725 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:36.290823 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:36.290932 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 14:58:36.291007 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 14:58:36.291769 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 18999
I20260806 14:58:36.305682 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 19176
I20260806 14:58:36.319504 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 19354
I20260806 14:58:36.328145 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 18877
2026-08-06T14:58:36Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2102 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 14:58:36 dist-test-slave-68xk krb5kdc[19575](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 14:58:36 dist-test-slave-68xk krb5kdc[19575](info): set up 2 sockets
Aug 06 14:58:36 dist-test-slave-68xk krb5kdc[19575](info): commencing operation
krb5kdc: starting...
W20260806 14:58:37.632195 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 14:58:37 dist-test-slave-68xk krb5kdc[19575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T14:58:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T14:58:37Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.126.keytab.
Entry for principal oryx/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.126.keytab.
I20260806 14:58:37.754624 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:34611
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.126.keytab
--principal=oryx/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:36345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:34611
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:37.840562 19593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:37.840782 19593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:37.840806 19593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:37.842072 19593 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 14:58:37.842124 19593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:37.842139 19593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:37.842152 19593 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 14:58:37.842164 19593 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 14:58:37.843694 19593 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:36345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:34611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.126.keytab
--principal=oryx/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:34611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:37.843950 19593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:37.844192 19593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:37.849131 19605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:37.849138 19603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:37.849208 19607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:37.849339 19593 server_base.cc:1034] running on GCE node
I20260806 14:58:37.849577 19593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:37.849890 19593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:37.851064 19593 hybrid_clock.cc:648] HybridClock initialized: now 1786028317851061 us; error 45 us; skew 500 ppm
Aug 06 14:58:37 dist-test-slave-68xk krb5kdc[19575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028317, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:37.853710 19593 init.cc:377] Logged in from keytab as oryx/127.11.8.126@KRBTEST.COM (short username oryx)
I20260806 14:58:37.854233 19593 webserver.cc:460] Webserver started at http://127.11.8.126:44861/ using document root <none> and password file <none>
I20260806 14:58:37.854542 19593 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:37.854619 19593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:37.854789 19593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:37.855873 19593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "d5ce68764a334405bdb27775b2321b6a"
format_stamp: "Formatted at 2026-08-06 14:58:37 on dist-test-slave-68xk"
server_key: "0791443f14d9f5bd959ee6ee64ebb9c1"
server_key_iv: "c34883fad25c16a78f2b5bc66fdfc13f"
server_key_version: "encryptionkey@0"
I20260806 14:58:37.856259 19593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "d5ce68764a334405bdb27775b2321b6a"
format_stamp: "Formatted at 2026-08-06 14:58:37 on dist-test-slave-68xk"
server_key: "0791443f14d9f5bd959ee6ee64ebb9c1"
server_key_iv: "c34883fad25c16a78f2b5bc66fdfc13f"
server_key_version: "encryptionkey@0"
I20260806 14:58:37.857645 19593 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 14:58:37.858525 19616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:37.858724 19593 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 14:58:37.858830 19593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "d5ce68764a334405bdb27775b2321b6a"
format_stamp: "Formatted at 2026-08-06 14:58:37 on dist-test-slave-68xk"
server_key: "0791443f14d9f5bd959ee6ee64ebb9c1"
server_key_iv: "c34883fad25c16a78f2b5bc66fdfc13f"
server_key_version: "encryptionkey@0"
I20260806 14:58:37.858913 19593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:37.875455 19593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:37.877453 19593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:37.877628 19593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:37.881935 19593 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:34611
I20260806 14:58:37.881969 19683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:34611 every 8 connection(s)
I20260806 14:58:37.882346 19593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 14:58:37.883481 19684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:37.885798 19684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a: Bootstrap starting.
I20260806 14:58:37.886356 19684 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:37.886693 19684 log.cc:824] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:37.887413 19684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a: No bootstrap required, opened a new log
I20260806 14:58:37.889245 19684 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } }
I20260806 14:58:37.889390 19684 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:37.889413 19684 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5ce68764a334405bdb27775b2321b6a, State: Initialized, Role: FOLLOWER
I20260806 14:58:37.889549 19684 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } }
I20260806 14:58:37.889647 19684 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 14:58:37.889669 19684 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 14:58:37.889698 19684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:37.889993 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 19593
I20260806 14:58:37.890117 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 14:58:37.890354 11297 external_mini_cluster.cc:1428] Setting key 2dbb6e153ef3df97bfb4ccc44ec193eb
I20260806 14:58:37.890865 19684 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } }
I20260806 14:58:37.891011 19684 leader_election.cc:302] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5ce68764a334405bdb27775b2321b6a; no voters:
I20260806 14:58:37.891244 19684 leader_election.cc:288] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 14:58:37.891290 19690 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:37.891474 19690 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [term 1 LEADER]: Becoming Leader. State: Replica: d5ce68764a334405bdb27775b2321b6a, State: Running, Role: LEADER
I20260806 14:58:37.891515 19684 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 14:58:37.891652 19690 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } }
I20260806 14:58:37.892140 19691 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5ce68764a334405bdb27775b2321b6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } }
I20260806 14:58:37.892225 19691 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:37.892397 19692 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5ce68764a334405bdb27775b2321b6a. Latest consensus state: current_term: 1 leader_uuid: "d5ce68764a334405bdb27775b2321b6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } }
I20260806 14:58:37.892454 19692 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:37.892779 19702 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 14:58:37.893347 19702 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 14:58:37.895145 19702 catalog_manager.cc:1347] Generated new cluster ID: c461a7c22dcb49a6a9a5e229a9598c26
I20260806 14:58:37.895200 19702 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 14:58:37 dist-test-slave-68xk krb5kdc[19575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM
I20260806 14:58:37.903192 19689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:37.890933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49752 (local address 127.11.8.126:34611)
0806 14:58:37.891112 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:37.891123 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:37.891162 (+ 39us) server_negotiation.cc:407] Connection header received
0806 14:58:37.891300 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:37.891307 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:37.891368 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:37.892386 (+ 1018us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:37.892514 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:37.893976 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:37.895009 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:37.895644 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:37.895755 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:37.898672 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:37.898676 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:37.898677 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:37.898690 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:37.901437 (+ 2747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:37.901844 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:37.901848 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:37.901849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:37.901974 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:37.902186 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:37.902187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:37.902188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:37.902559 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:37.902744 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:37.902820 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 14:58:37.902970 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
I20260806 14:58:37.908399 19702 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 14:58:37.908906 19702 catalog_manager.cc:1504] Loading token signing keys...
I20260806 14:58:37.911214 19702 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a: Generated new TSK 0
I20260806 14:58:37.911406 19702 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.65.keytab.
Entry for principal oryx/127.11.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.65.keytab.
I20260806 14:58:37.943488 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:0
--local_ip_for_outbound_sockets=127.11.8.65
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:34611
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.65.keytab
--principal=oryx/127.11.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:36345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:38.028644 19723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:38.028848 19723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:38.028872 19723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:38.030010 19723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:38.030063 19723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:38.030089 19723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 14:58:38.031569 19723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:36345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.65.keytab
--principal=oryx/127.11.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:34611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:38.031817 19723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:38.032078 19723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:38.037266 19730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:38.037236 19731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:38.037241 19733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:38.037634 19723 server_base.cc:1034] running on GCE node
I20260806 14:58:38.037861 19723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:38.038141 19723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:38.039316 19723 hybrid_clock.cc:648] HybridClock initialized: now 1786028318039298 us; error 36 us; skew 500 ppm
Aug 06 14:58:38 dist-test-slave-68xk krb5kdc[19575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028318, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:38.042008 19723 init.cc:377] Logged in from keytab as oryx/127.11.8.65@KRBTEST.COM (short username oryx)
I20260806 14:58:38.042579 19723 webserver.cc:460] Webserver started at http://127.11.8.65:46109/ using document root <none> and password file <none>
I20260806 14:58:38.042837 19723 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:38.042904 19723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:38.043056 19723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:38.044114 19723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/data/instance:
uuid: "9e45adfc7114482c9bfd1cc0b6b75a96"
format_stamp: "Formatted at 2026-08-06 14:58:38 on dist-test-slave-68xk"
server_key: "2b7f956a6bf6230bb24d98ef4caf4f4a"
server_key_iv: "e03d38b1add7b04b2963a56a15d132ad"
server_key_version: "encryptionkey@0"
I20260806 14:58:38.044471 19723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance:
uuid: "9e45adfc7114482c9bfd1cc0b6b75a96"
format_stamp: "Formatted at 2026-08-06 14:58:38 on dist-test-slave-68xk"
server_key: "2b7f956a6bf6230bb24d98ef4caf4f4a"
server_key_iv: "e03d38b1add7b04b2963a56a15d132ad"
server_key_version: "encryptionkey@0"
I20260806 14:58:38.045825 19723 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 14:58:38.046609 19742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:38.046790 19723 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:38.046891 19723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "9e45adfc7114482c9bfd1cc0b6b75a96"
format_stamp: "Formatted at 2026-08-06 14:58:38 on dist-test-slave-68xk"
server_key: "2b7f956a6bf6230bb24d98ef4caf4f4a"
server_key_iv: "e03d38b1add7b04b2963a56a15d132ad"
server_key_version: "encryptionkey@0"
I20260806 14:58:38.046970 19723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:38.066092 19723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:38.068099 19723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:38.068271 19723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:38.068509 19723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:38.068835 19723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:38.068890 19723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:38.068931 19723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:38.068959 19723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:38.074955 19723 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:40721
I20260806 14:58:38.075009 19884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:40721 every 8 connection(s)
I20260806 14:58:38.075320 19723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
I20260806 14:58:38.079588 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 19723
I20260806 14:58:38.079694 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance
I20260806 14:58:38.079909 11297 external_mini_cluster.cc:1428] Setting key 0155bf4041dc09219867b2c566856560
Aug 06 14:58:38 dist-test-slave-68xk krb5kdc[19575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028318, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM
I20260806 14:58:38.085614 19689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.076050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:41397 (local address 127.11.8.126:34611)
0806 14:58:38.076122 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:38.076124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:38.076337 (+ 213us) server_negotiation.cc:407] Connection header received
0806 14:58:38.076869 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:38.076871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:38.076886 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:38.076949 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:38.077874 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.078505 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.079713 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.080050 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.080152 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.082661 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:38.082662 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:38.082663 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:38.082672 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:38.084163 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.084598 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.084600 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.084601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.084664 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.084918 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.084919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.084919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.085050 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:38.085135 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:38.085412 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 14:58:38.085514 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 14:58:38.085795 19888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.076033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.11.8.65:41397)
0806 14:58:38.076199 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:38.076244 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:38.076747 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:38.077001 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:38.077004 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:38.077562 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:38.077762 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.077767 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.078655 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.078656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.079576 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.079579 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.080163 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.080164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.080324 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.080860 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:38.080873 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:38.082559 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:38.084272 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.084280 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.084282 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.084512 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.084734 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.084735 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.084736 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.084850 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.085127 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:38.085134 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:38.085277 (+ 143us) client_negotiation.cc:769] Sending connection context
0806 14:58:38.085390 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 14:58:38.085556 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":88,"threads_started":1}
I20260806 14:58:38.086304 19885 heartbeater.cc:347] Connected to a master server at 127.11.8.126:34611
I20260806 14:58:38.086445 19885 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:38.086648 19885 heartbeater.cc:511] Master 127.11.8.126:34611 requested a full tablet report, sending...
I20260806 14:58:38.087209 19637 ts_manager.cc:194] Registered new tserver with Master: 9e45adfc7114482c9bfd1cc0b6b75a96 (127.11.8.65:40721)
I20260806 14:58:38.087852 19637 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.8.65@KRBTEST.COM'} at 127.11.8.65:41397
WARNING: no policy specified for oryx/127.11.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.66.keytab.
Entry for principal oryx/127.11.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.66.keytab.
I20260806 14:58:38.107456 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:0
--local_ip_for_outbound_sockets=127.11.8.66
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:34611
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.66.keytab
--principal=oryx/127.11.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:36345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:38.199373 19891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:38.199620 19891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:38.199648 19891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:38.200927 19891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:38.200974 19891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:38.201004 19891 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 14:58:38.202755 19891 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:36345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.66.keytab
--principal=oryx/127.11.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:34611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:38.203029 19891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:38.203311 19891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:38.208927 19898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:38.208894 19900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:38.208894 19904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:38.209029 19891 server_base.cc:1034] running on GCE node
I20260806 14:58:38.209421 19891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:38.209738 19891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:38.210920 19891 hybrid_clock.cc:648] HybridClock initialized: now 1786028318210897 us; error 38 us; skew 500 ppm
Aug 06 14:58:38 dist-test-slave-68xk krb5kdc[19575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028318, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:38.213586 19891 init.cc:377] Logged in from keytab as oryx/127.11.8.66@KRBTEST.COM (short username oryx)
I20260806 14:58:38.214010 19891 webserver.cc:460] Webserver started at http://127.11.8.66:38151/ using document root <none> and password file <none>
I20260806 14:58:38.214260 19891 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:38.214334 19891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:38.214512 19891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:38.215565 19891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/data/instance:
uuid: "cd4e5e034bb647b2adf0bbf9f4850843"
format_stamp: "Formatted at 2026-08-06 14:58:38 on dist-test-slave-68xk"
server_key: "d3a842c99ec0ed2c4edd38261e12cb73"
server_key_iv: "55798dc5e1a9e3d1be762676adfdfab4"
server_key_version: "encryptionkey@0"
I20260806 14:58:38.215929 19891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance:
uuid: "cd4e5e034bb647b2adf0bbf9f4850843"
format_stamp: "Formatted at 2026-08-06 14:58:38 on dist-test-slave-68xk"
server_key: "d3a842c99ec0ed2c4edd38261e12cb73"
server_key_iv: "55798dc5e1a9e3d1be762676adfdfab4"
server_key_version: "encryptionkey@0"
I20260806 14:58:38.217262 19891 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 14:58:38.218012 19912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:38.218199 19891 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 14:58:38.218304 19891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "cd4e5e034bb647b2adf0bbf9f4850843"
format_stamp: "Formatted at 2026-08-06 14:58:38 on dist-test-slave-68xk"
server_key: "d3a842c99ec0ed2c4edd38261e12cb73"
server_key_iv: "55798dc5e1a9e3d1be762676adfdfab4"
server_key_version: "encryptionkey@0"
I20260806 14:58:38.218390 19891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:38.236930 19891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:38.239753 19891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:38.239925 19891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:38.240203 19891 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:38.240546 19891 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:38.240605 19891 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:38.240641 19891 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:38.240671 19891 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:38.246783 19891 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:46791
I20260806 14:58:38.246837 20066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:46791 every 8 connection(s)
I20260806 14:58:38.247195 19891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
I20260806 14:58:38.252868 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 19891
I20260806 14:58:38.253008 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance
I20260806 14:58:38.253243 11297 external_mini_cluster.cc:1428] Setting key f98268e3b4eac70664f7120c3438e159
Aug 06 14:58:38 dist-test-slave-68xk krb5kdc[19575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028318, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM
I20260806 14:58:38.257201 19689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.247929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:33979 (local address 127.11.8.126:34611)
0806 14:58:38.248016 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:38.248018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:38.248228 (+ 210us) server_negotiation.cc:407] Connection header received
0806 14:58:38.248795 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:38.248796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:38.248810 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:38.248866 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:38.249902 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.250483 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.251564 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.251786 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.251867 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.254285 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:38.254287 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:38.254287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:38.254296 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:38.255673 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.256195 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.256196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.256196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.256274 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.256566 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.256567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.256567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.256674 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:38.256727 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:38.257062 (+ 335us) server_negotiation.cc:299] Negotiation successful
0806 14:58:38.257137 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 14:58:38.257512 20069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.247884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.11.8.66:33979)
0806 14:58:38.248085 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:38.248132 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:38.248675 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:38.248938 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:38.248943 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:38.249546 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:38.249804 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.249809 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.250590 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.250591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.251464 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.251467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.251876 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.251877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.252008 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.252519 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:38.252527 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:38.254191 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:38.255758 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.255766 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.255768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.256099 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.256351 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.256352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.256352 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.256499 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.256740 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:38.256746 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:38.256915 (+ 169us) client_negotiation.cc:769] Sending connection context
0806 14:58:38.257045 (+ 130us) client_negotiation.cc:240] Negotiation successful
0806 14:58:38.257239 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":99,"threads_started":1}
I20260806 14:58:38.258062 20067 heartbeater.cc:347] Connected to a master server at 127.11.8.126:34611
I20260806 14:58:38.258157 20067 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:38.258352 20067 heartbeater.cc:511] Master 127.11.8.126:34611 requested a full tablet report, sending...
I20260806 14:58:38.258787 19637 ts_manager.cc:194] Registered new tserver with Master: cd4e5e034bb647b2adf0bbf9f4850843 (127.11.8.66:46791)
I20260806 14:58:38.259346 19637 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.8.66@KRBTEST.COM'} at 127.11.8.66:33979
WARNING: no policy specified for oryx/127.11.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.67.keytab.
Entry for principal oryx/127.11.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.67.keytab.
I20260806 14:58:38.280805 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:0
--local_ip_for_outbound_sockets=127.11.8.67
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:34611
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.67.keytab
--principal=oryx/127.11.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:36345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:38.364048 20076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:38.364248 20076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:38.364270 20076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:38.365353 20076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:38.365391 20076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:38.365417 20076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 14:58:38.366814 20076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:36345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.67.keytab
--principal=oryx/127.11.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:34611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:38.367009 20076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:38.367189 20076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:38.372160 20086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:38.372160 20083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:38.372172 20084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:38.372301 20076 server_base.cc:1034] running on GCE node
I20260806 14:58:38.372524 20076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:38.372803 20076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:38.373956 20076 hybrid_clock.cc:648] HybridClock initialized: now 1786028318373928 us; error 45 us; skew 500 ppm
Aug 06 14:58:38 dist-test-slave-68xk krb5kdc[19575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028318, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:38.376566 20076 init.cc:377] Logged in from keytab as oryx/127.11.8.67@KRBTEST.COM (short username oryx)
I20260806 14:58:38.377002 20076 webserver.cc:460] Webserver started at http://127.11.8.67:34807/ using document root <none> and password file <none>
I20260806 14:58:38.377216 20076 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:38.377257 20076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:38.377343 20076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:38.378304 20076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/data/instance:
uuid: "7887589cb7404641939ec7d93266fcca"
format_stamp: "Formatted at 2026-08-06 14:58:38 on dist-test-slave-68xk"
server_key: "91c7fc0564b815216df850b095aae6f1"
server_key_iv: "d759ecafd136cda2b19182947074bf76"
server_key_version: "encryptionkey@0"
I20260806 14:58:38.378804 20076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance:
uuid: "7887589cb7404641939ec7d93266fcca"
format_stamp: "Formatted at 2026-08-06 14:58:38 on dist-test-slave-68xk"
server_key: "91c7fc0564b815216df850b095aae6f1"
server_key_iv: "d759ecafd136cda2b19182947074bf76"
server_key_version: "encryptionkey@0"
I20260806 14:58:38.380038 20076 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:38.380724 20095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:38.380911 20076 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:38.381009 20076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "7887589cb7404641939ec7d93266fcca"
format_stamp: "Formatted at 2026-08-06 14:58:38 on dist-test-slave-68xk"
server_key: "91c7fc0564b815216df850b095aae6f1"
server_key_iv: "d759ecafd136cda2b19182947074bf76"
server_key_version: "encryptionkey@0"
I20260806 14:58:38.381088 20076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:38.391733 20076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:38.394256 20076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:38.394460 20076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:38.394685 20076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:38.395001 20076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:38.395035 20076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:38.395087 20076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:38.395113 20076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:38.400748 20076 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:34831
I20260806 14:58:38.400825 20257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:34831 every 8 connection(s)
I20260806 14:58:38.401140 20076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
I20260806 14:58:38.405871 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 20076
I20260806 14:58:38.405972 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance
I20260806 14:58:38.406152 11297 external_mini_cluster.cc:1428] Setting key bbedd62f4e923f0b47d27a9abf80ccdb
Aug 06 14:58:38 dist-test-slave-68xk krb5kdc[19575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028318, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM
I20260806 14:58:38.410295 19689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.401879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:33709 (local address 127.11.8.126:34611)
0806 14:58:38.401946 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:38.401948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:38.402180 (+ 232us) server_negotiation.cc:407] Connection header received
0806 14:58:38.402714 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:38.402715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:38.402725 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:38.402786 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:38.403684 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.404180 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.405186 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.405389 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.405455 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.407973 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:38.407974 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:38.407974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:38.407982 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:38.409163 (+ 1181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.409526 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.409527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.409527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.409579 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.409787 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.409787 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.409788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.409884 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:38.409933 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:38.410151 (+ 218us) server_negotiation.cc:299] Negotiation successful
0806 14:58:38.410219 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 14:58:38.410480 20260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.401847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.11.8.67:33709)
0806 14:58:38.402038 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:38.402081 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:38.402628 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:38.402839 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:38.402842 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:38.403404 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:38.403602 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.403606 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.404264 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.404264 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.405075 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.405078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.405472 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.405474 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.405591 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.406195 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:38.406203 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:38.407868 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:38.409241 (+ 1373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.409247 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.409248 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.409458 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.409638 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.409639 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.409639 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.409730 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.409937 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:38.409941 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:38.410051 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 14:58:38.410130 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 14:58:38.410263 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":103,"threads_started":1}
I20260806 14:58:38.410949 20258 heartbeater.cc:347] Connected to a master server at 127.11.8.126:34611
I20260806 14:58:38.411056 20258 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:38.411249 20258 heartbeater.cc:511] Master 127.11.8.126:34611 requested a full tablet report, sending...
I20260806 14:58:38.411634 19637 ts_manager.cc:194] Registered new tserver with Master: 7887589cb7404641939ec7d93266fcca (127.11.8.67:34831)
I20260806 14:58:38.412009 19637 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.8.67@KRBTEST.COM'} at 127.11.8.67:33709
I20260806 14:58:38.418306 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.8.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.8.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.8.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.125.keytab.
Entry for principal oryx/127.11.8.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.125.keytab.
I20260806 14:58:38.445925 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.125:42909
--webserver_interface=127.11.8.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.125.keytab
--principal=oryx/127.11.8.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:36345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:34611,127.11.8.125:42909
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:38.531744 20264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:38.531953 20264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:38.531977 20264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:38.533152 20264 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 14:58:38.533196 20264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:38.533211 20264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:38.533224 20264 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 14:58:38.533278 20264 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 14:58:38.534758 20264 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:36345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:34611,127.11.8.125:42909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.125.keytab
--principal=oryx/127.11.8.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.125:42909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.8.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
W20260806 14:58:38.534862 20264 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.11.8.126:34611,127.11.8.125:42909'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 14:58:38.535028 20264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:38.535282 20264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:38.540179 20280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:38.540184 20275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:38.540418 20264 server_base.cc:1034] running on GCE node
W20260806 14:58:38.540154 20278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:38.540782 20264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:38.541095 20264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:38.542294 20264 hybrid_clock.cc:648] HybridClock initialized: now 1786028318542273 us; error 34 us; skew 500 ppm
Aug 06 14:58:38 dist-test-slave-68xk krb5kdc[19575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028318, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:38.545039 20264 init.cc:377] Logged in from keytab as oryx/127.11.8.125@KRBTEST.COM (short username oryx)
I20260806 14:58:38.545557 20264 webserver.cc:460] Webserver started at http://127.11.8.125:44245/ using document root <none> and password file <none>
I20260806 14:58:38.545768 20264 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:38.545810 20264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:38.545900 20264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:38.547041 20264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-1/data/instance:
uuid: "ef653e0dfa3841999c9d8f70b12f03e6"
format_stamp: "Formatted at 2026-08-06 14:58:38 on dist-test-slave-68xk"
server_key: "83aeae829878a076dffa0fffbd66e8a4"
server_key_iv: "d4e7503a77b4da8b99bfb78e146082cd"
server_key_version: "encryptionkey@0"
I20260806 14:58:38.547348 20264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-1/wal/instance:
uuid: "ef653e0dfa3841999c9d8f70b12f03e6"
format_stamp: "Formatted at 2026-08-06 14:58:38 on dist-test-slave-68xk"
server_key: "83aeae829878a076dffa0fffbd66e8a4"
server_key_iv: "d4e7503a77b4da8b99bfb78e146082cd"
server_key_version: "encryptionkey@0"
I20260806 14:58:38.548616 20264 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 14:58:38.549360 20291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:38.549536 20264 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:38.549650 20264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-1/wal
uuid: "ef653e0dfa3841999c9d8f70b12f03e6"
format_stamp: "Formatted at 2026-08-06 14:58:38 on dist-test-slave-68xk"
server_key: "83aeae829878a076dffa0fffbd66e8a4"
server_key_iv: "d4e7503a77b4da8b99bfb78e146082cd"
server_key_version: "encryptionkey@0"
I20260806 14:58:38.549738 20264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:38.561584 20264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:38.563774 20264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:38.563934 20264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:38.568048 20264 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.125:42909
I20260806 14:58:38.568084 20361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.125:42909 every 8 connection(s)
I20260806 14:58:38.568435 20264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-1/data/info.pb
I20260806 14:58:38.569443 20362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:38.571002 20362 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } has no permanent_uuid. Determining permanent_uuid...
I20260806 14:58:38.571245 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 20264
Aug 06 14:58:38 dist-test-slave-68xk krb5kdc[19575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028318, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.125@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM
Aug 06 14:58:38 dist-test-slave-68xk krb5kdc[19575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.8.125@KRBTEST.COM
I20260806 14:58:38.581022 19689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.571823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49766 (local address 127.11.8.126:34611)
0806 14:58:38.571883 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:38.571885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:38.572119 (+ 234us) server_negotiation.cc:407] Connection header received
0806 14:58:38.572724 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:38.572725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:38.572739 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:38.572808 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:38.573321 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.573861 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.575023 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.575331 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.575404 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.577939 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:38.577940 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:38.577941 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:38.577949 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:38.579458 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.579943 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.579944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.579944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.579999 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.580285 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.580286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.580286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.580428 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:38.580480 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:38.580832 (+ 352us) server_negotiation.cc:299] Negotiation successful
0806 14:58:38.580913 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 14:58:38.581055 20365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.571759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.0.0.1:49766)
0806 14:58:38.571975 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:38.572021 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:38.572588 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:38.572844 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:38.572845 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:38.573073 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:38.573217 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.573221 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.573956 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.573957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.574938 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.574941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.575416 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.575417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.575546 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.576113 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:38.576121 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:38.577843 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:38.579540 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.579548 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.579550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.579852 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.580084 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.580085 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.580085 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.580213 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.580498 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:38.580503 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:38.580664 (+ 161us) client_negotiation.cc:769] Sending connection context
0806 14:58:38.580774 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 14:58:38.580896 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":97,"threads_started":1}
I20260806 14:58:38.581806 20362 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } has no permanent_uuid. Determining permanent_uuid...
I20260806 14:58:38.582132 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.571998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42536 (local address 127.11.8.125:42909)
0806 14:58:38.572188 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:38.572192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:38.572221 (+ 29us) server_negotiation.cc:407] Connection header received
0806 14:58:38.572590 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:38.572602 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:38.572629 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:38.572716 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:38.573441 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.574515 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.575406 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.575821 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.575875 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.578797 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:38.578804 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:38.578804 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:38.578811 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:38.580670 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.581174 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.581178 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.581178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.581240 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.581436 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.581437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.581437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.581635 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:38.581781 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:38.581919 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 14:58:38.582001 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":88,"threads_started":1}
Aug 06 14:58:38 dist-test-slave-68xk krb5kdc[19575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028318, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.125@KRBTEST.COM for oryx/127.11.8.125@KRBTEST.COM
I20260806 14:58:38.588449 20365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.582055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:42909 (local address 127.0.0.1:42552)
0806 14:58:38.582180 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:38.582191 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:38.582236 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:38.582460 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:38.582460 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:38.582619 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:38.582700 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.582702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.583344 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.583345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.583914 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.583916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.584281 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.584282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.584335 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.584755 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:38.584759 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:38.585984 (+ 1225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:38.587668 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.587668 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.587669 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.587828 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.587998 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.587999 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.587999 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.588041 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.588242 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:38.588243 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:38.588274 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 14:58:38.588320 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 14:58:38.588390 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109}
I20260806 14:58:38.588510 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.582089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42552 (local address 127.11.8.125:42909)
0806 14:58:38.582256 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:38.582259 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:38.582269 (+ 10us) server_negotiation.cc:407] Connection header received
0806 14:58:38.582290 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:38.582291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:38.582301 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:38.582353 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:38.582791 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.583265 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.583987 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.584189 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.584250 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.586073 (+ 1823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:38.586074 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:38.586074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:38.586081 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:38.587572 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.587893 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.587894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.587894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.587944 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.588096 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.588097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.588097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.588198 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:38.588250 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:38.588353 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 14:58:38.588435 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":151,"thread_start_us":87,"threads_started":1}
I20260806 14:58:38.589756 20362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6: Bootstrap starting.
I20260806 14:58:38.590765 20362 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:38.591051 20362 log.cc:824] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:38.591899 20362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6: No bootstrap required, opened a new log
I20260806 14:58:38.593554 20362 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } }
I20260806 14:58:38.593686 20362 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:38.593734 20362 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef653e0dfa3841999c9d8f70b12f03e6, State: Initialized, Role: FOLLOWER
I20260806 14:58:38.593891 20362 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } }
I20260806 14:58:38.594182 20362 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 14:58:38.594152 20372 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } } }
I20260806 14:58:38.594255 20372 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 14:58:38.596066 20389 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 14:58:38.596130 20389 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 14:58:38.598042 20264 master_runner.cc:428] Detected that this master ef653e0dfa3841999c9d8f70b12f03e6 is joining an existing cluster
I20260806 14:58:38.598127 20264 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.8.125:42909
I20260806 14:58:38.607234 19689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.599775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49782 (local address 127.11.8.126:34611)
0806 14:58:38.599846 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:38.599848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:38.599943 (+ 95us) server_negotiation.cc:407] Connection header received
0806 14:58:38.600049 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:38.600050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:38.600064 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:38.600126 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:38.600580 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.601229 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.602095 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.602313 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.602441 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.603466 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:38.603467 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:38.603468 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:38.603475 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:38.605132 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.606370 (+ 1238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.606372 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.606372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.606451 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.606653 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.606655 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.606655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.606791 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:38.606868 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:38.607036 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 14:58:38.607139 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 14:58:38.607247 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.600003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42558 (local address 127.11.8.125:42909)
0806 14:58:38.600043 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:38.600045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:38.600259 (+ 214us) server_negotiation.cc:407] Connection header received
0806 14:58:38.600334 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:38.600335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:38.600346 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:38.600427 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:38.600834 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.601542 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.602564 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.603161 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.603207 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.604338 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:38.604339 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:38.604340 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:38.604346 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:38.605951 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.606469 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.606470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.606470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.606524 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.606704 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.606705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.606705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.606817 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:38.606883 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:38.607071 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 14:58:38.607142 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 14:58:38.607407 20396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.599722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.0.0.1:49782)
0806 14:58:38.599868 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:38.599879 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:38.599970 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:38.600171 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:38.600173 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:38.600390 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:38.600489 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.600491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.601325 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.601326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.601970 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.601972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.602454 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.602455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.602524 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.603025 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:38.603036 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:38.603333 (+ 297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:38.605233 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.605234 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.605234 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.606287 (+ 1053us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.606518 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.606518 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.606519 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.606577 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.606868 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:38.606869 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:38.606914 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 14:58:38.606991 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 14:58:38.607090 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
I20260806 14:58:38.607861 20397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.599943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:42909 (local address 127.0.0.1:42558)
0806 14:58:38.600180 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:38.600191 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:38.600272 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:38.600431 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:38.600431 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:38.600686 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:38.600745 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.600747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.601635 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.601636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.602473 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.602475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.603288 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.603289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.603346 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.603861 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:38.603866 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:38.604234 (+ 368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:38.606072 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.606073 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.606073 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.606359 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.606594 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.606595 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.606595 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.606648 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.606905 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:38.606906 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:38.606941 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 14:58:38.607644 (+ 703us) client_negotiation.cc:240] Negotiation successful
0806 14:58:38.607762 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":168,"threads_started":1}
Aug 06 14:58:38 dist-test-slave-68xk krb5kdc[19575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028317, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.126@KRBTEST.COM for oryx/127.11.8.125@KRBTEST.COM
I20260806 14:58:38.617538 20398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.610038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:42909 (local address 127.0.0.1:42566)
0806 14:58:38.610178 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:38.610226 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:38.610336 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:38.610497 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:38.610501 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:38.610787 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:38.610832 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.610833 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.611453 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.611454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.612037 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.612038 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.612421 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.612423 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.612501 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.612997 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:38.613001 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:38.614544 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:38.616157 (+ 1613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.616164 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.616166 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.616463 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.616691 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.616692 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.616692 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.616794 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.617114 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:38.617122 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:38.617191 (+ 69us) client_negotiation.cc:769] Sending connection context
0806 14:58:38.617298 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 14:58:38.617431 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":56,"threads_started":1}
I20260806 14:58:38.617595 20368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.610165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42566 (local address 127.11.8.125:42909)
0806 14:58:38.610205 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:38.610207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:38.610309 (+ 102us) server_negotiation.cc:407] Connection header received
0806 14:58:38.610431 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:38.610432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:38.610443 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:38.610493 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:38.610924 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.611366 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.612126 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.612331 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.612417 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.614654 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:38.614655 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:38.614655 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:38.614662 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:38.616016 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.616531 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.616533 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.616533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.616612 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.616870 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.616872 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.616872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.617018 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:38.617106 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:38.617346 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 14:58:38.617467 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 14:58:38.618546 19637 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.8.125:42909: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6"
member_type: NON_VOTER
last_known_addr {
host: "127.11.8.125"
port: 42909
}
attrs {
promote: true
}
}
dest_uuid: "d5ce68764a334405bdb27775b2321b6a"
cas_config_opid_index: -1
I20260806 14:58:38.619027 19637 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: NON_VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: true } }
I20260806 14:58:38.619633 20399 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER ef653e0dfa3841999c9d8f70b12f03e6 (127.11.8.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: NON_VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: true } } }
I20260806 14:58:38.620126 20327 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:38.621330 20327 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 1 FOLLOWER]: Refusing update from remote peer d5ce68764a334405bdb27775b2321b6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 14:58:38.621758 20401 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: NON_VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:38.622514 20399 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.8.125:42909
I20260806 14:58:38.622850 20264 master.cc:537] Master@127.11.8.125:42909 shutting down...
I20260806 14:58:38.623893 20327 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER ef653e0dfa3841999c9d8f70b12f03e6 (127.11.8.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: NON_VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: true } } }
I20260806 14:58:38.625223 20405 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d5ce68764a334405bdb27775b2321b6a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: NON_VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: true } } }
I20260806 14:58:38.625236 20372 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5ce68764a334405bdb27775b2321b6a. Latest consensus state: current_term: 1 leader_uuid: "d5ce68764a334405bdb27775b2321b6a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: NON_VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: true } } }
I20260806 14:58:38.625332 20405 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [sys.catalog]: This master's current role is: LEARNER
I20260806 14:58:38.625355 20372 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [sys.catalog]: This master's current role is: LEARNER
I20260806 14:58:38.625368 20405 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d5ce68764a334405bdb27775b2321b6a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: NON_VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: true } } }
I20260806 14:58:38.625398 20405 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [sys.catalog]: This master's current role is: LEARNER
I20260806 14:58:38.626966 20403 mvcc.cc:205] Tried to move back new op lower bound from 7315571990162898944 to 7315571990085001216. Current Snapshot: MvccSnapshot[applied={T|T < 7315571990095044608}]
I20260806 14:58:38.627406 20401 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d5ce68764a334405bdb27775b2321b6a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: NON_VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: true } } }
I20260806 14:58:38.627526 20401 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:38.633960 20264 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 1 LEARNER]: Raft consensus shutting down.
I20260806 14:58:38.634105 20264 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 1 LEARNER]: Raft consensus is shut down!
I20260806 14:58:38.634155 20264 tablet_replica.cc:330] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6: stopping tablet replica
W20260806 14:58:38.635612 19572 connection.cc:535] client connection to 127.11.8.125:42909 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 14:58:38.648063 20264 master.cc:552] Master@127.11.8.125:42909 shutdown complete.
I20260806 14:58:38.648144 20264 master_runner.cc:305] Clearing existing system tablet
I20260806 14:58:38.648293 20264 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:38.648365 20264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:38.649130 20264 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:38.649780 20415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:38.649924 20264 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:38.649997 20264 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 14:58:38.650025 20264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-1/wal
uuid: "ef653e0dfa3841999c9d8f70b12f03e6"
format_stamp: "Formatted at 2026-08-06 14:58:38 on dist-test-slave-68xk"
server_key: "83aeae829878a076dffa0fffbd66e8a4"
server_key_iv: "d4e7503a77b4da8b99bfb78e146082cd"
server_key_version: "encryptionkey@0"
I20260806 14:58:38.651021 20264 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:38.652578 20264 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 14:58:38.652652 20264 log.cc:1196] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 14:58:38.652902 20264 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6: Deleting consensus metadata
I20260806 14:58:38.653035 20264 master_runner.cc:315] Copying system tablet from 127.11.8.126:34611
I20260806 14:58:38.653648 20264 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6: tablet copy: Beginning tablet copy session from remote peer at address 127.11.8.126:34611
I20260806 14:58:38.660681 19689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.654144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49788 (local address 127.11.8.126:34611)
0806 14:58:38.654218 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:38.654221 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:38.654324 (+ 103us) server_negotiation.cc:407] Connection header received
0806 14:58:38.654404 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:38.654405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:38.654489 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:38.654566 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:38.654950 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.655563 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.656629 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.656830 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.656891 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.657964 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:38.657965 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:38.657965 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:38.657971 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:38.659583 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.659968 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.659968 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.659969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.660021 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.660218 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.660218 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.660218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.660319 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:38.660374 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:38.660527 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 14:58:38.660616 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 14:58:38.660703 20421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.654086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.0.0.1:49788)
0806 14:58:38.654257 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:38.654269 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:38.654339 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:38.654594 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:38.654595 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:38.654793 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:38.654869 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.654871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.655641 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.655642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.656542 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.656544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.656906 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.656907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.656975 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.657479 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:38.657485 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:38.657879 (+ 394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:38.659665 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.659666 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.659666 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.659890 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.660086 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.660086 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.660087 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.660154 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.660382 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:38.660384 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:38.660429 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 14:58:38.660507 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 14:58:38.660607 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":106,"threads_started":1}
I20260806 14:58:38.661464 19662 tablet_copy_service.cc:140] P d5ce68764a334405bdb27775b2321b6a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ef653e0dfa3841999c9d8f70b12f03e6 ({username='oryx', principal='oryx/127.11.8.125@KRBTEST.COM'} at 127.0.0.1:49788)
I20260806 14:58:38.661607 19662 tablet_copy_service.cc:161] P d5ce68764a334405bdb27775b2321b6a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ef653e0dfa3841999c9d8f70b12f03e6 at {username='oryx', principal='oryx/127.11.8.125@KRBTEST.COM'} at 127.0.0.1:49788: session id = ef653e0dfa3841999c9d8f70b12f03e6-00000000000000000000000000000000
I20260806 14:58:38.662514 19662 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 14:58:38.663159 20264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 14:58:38.664301 20264 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6: tablet copy: Starting download of 0 data blocks...
I20260806 14:58:38.664443 20264 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6: tablet copy: Starting download of 1 WAL segments...
I20260806 14:58:38.665580 20264 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 14:58:38.666503 19662 tablet_copy_service.cc:342] P d5ce68764a334405bdb27775b2321b6a: Request end of tablet copy session ef653e0dfa3841999c9d8f70b12f03e6-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.8.125@KRBTEST.COM'} at 127.0.0.1:49788
I20260806 14:58:38.666595 19662 tablet_copy_service.cc:433] P d5ce68764a334405bdb27775b2321b6a: ending tablet copy session ef653e0dfa3841999c9d8f70b12f03e6-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ef653e0dfa3841999c9d8f70b12f03e6
I20260806 14:58:38.669437 20264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:38.670657 20422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:38.670699 20426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:38.670734 20428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:38.670861 20264 server_base.cc:1034] running on GCE node
I20260806 14:58:38.670989 20264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:38.671137 20264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:38.672272 20264 hybrid_clock.cc:648] HybridClock initialized: now 1786028318672265 us; error 26 us; skew 500 ppm
Aug 06 14:58:38 dist-test-slave-68xk krb5kdc[19575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028318, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:38.673738 20264 init.cc:377] Logged in from keytab as oryx/127.11.8.125@KRBTEST.COM (short username oryx)
I20260806 14:58:38.674013 20264 webserver.cc:460] Webserver started at http://127.11.8.125:45895/ using document root <none> and password file <none>
I20260806 14:58:38.674144 20264 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:38.674183 20264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:38.674959 20264 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260806 14:58:38.675563 20441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:38.675740 20264 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 14:58:38.675798 20264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-1/wal
uuid: "ef653e0dfa3841999c9d8f70b12f03e6"
format_stamp: "Formatted at 2026-08-06 14:58:38 on dist-test-slave-68xk"
server_key: "83aeae829878a076dffa0fffbd66e8a4"
server_key_iv: "d4e7503a77b4da8b99bfb78e146082cd"
server_key_version: "encryptionkey@0"
I20260806 14:58:38.675853 20264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:38.706813 20264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:38.708581 20264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:38.711804 20264 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.125:42909
I20260806 14:58:38.711902 20506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.125:42909 every 8 connection(s)
I20260806 14:58:38.712164 20264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-1/data/info.pb
I20260806 14:58:38.713011 20507 sys_catalog.cc:263] Verifying existing consensus state
I20260806 14:58:38.713500 20507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6: Bootstrap starting.
I20260806 14:58:38.715672 20507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 14:58:38.715917 20507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6: Bootstrap complete.
I20260806 14:58:38.716164 20507 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: NON_VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: true } }
I20260806 14:58:38.716240 20507 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ef653e0dfa3841999c9d8f70b12f03e6, State: Initialized, Role: LEARNER
I20260806 14:58:38.716269 20507 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: NON_VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: true } }
I20260806 14:58:38.716487 20507 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 14:58:38.716516 20512 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: NON_VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: true } } }
I20260806 14:58:38.716614 20512 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [sys.catalog]: This master's current role is: LEARNER
I20260806 14:58:38.718156 20529 catalog_manager.cc:1259] Loaded cluster ID: c461a7c22dcb49a6a9a5e229a9598c26
I20260806 14:58:38.718247 20529 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6: loading cluster ID for follower catalog manager: success
I20260806 14:58:38.719250 20529 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6: acquiring CA information for follower catalog manager: success
I20260806 14:58:38.719563 20529 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 14:58:38.743608 20530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.736817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42584 (local address 127.11.8.125:42909)
0806 14:58:38.736958 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:38.736965 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:38.736973 (+ 8us) server_negotiation.cc:407] Connection header received
0806 14:58:38.736997 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:38.736997 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:38.737006 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:38.737103 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:38.737700 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.738403 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.739222 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.739444 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.739526 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.740828 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:38.740830 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:38.740830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:38.740838 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:38.742365 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.742758 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.742759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.742760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.742825 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.743029 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.743039 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.743039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.743161 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:38.743267 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:38.743378 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 14:58:38.743465 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
WARNING: no policy specified for oryx/127.11.8.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.8.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.8.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.124.keytab.
Entry for principal oryx/127.11.8.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.124.keytab.
I20260806 14:58:38.772699 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.124:34581
--webserver_interface=127.11.8.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.124.keytab
--principal=oryx/127.11.8.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:36345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:34611,127.11.8.125:42909,127.11.8.124:34581
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:38.856043 20534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:38.856254 20534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:38.856276 20534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:38.857376 20534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 14:58:38.857417 20534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:38.857431 20534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:38.857445 20534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 14:58:38.857456 20534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 14:58:38.859228 20534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:36345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:34611,127.11.8.125:42909,127.11.8.124:34581
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.124.keytab
--principal=oryx/127.11.8.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.124:34581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.8.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:38.859494 20534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:38.859709 20534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:38.864656 20542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:38.864660 20545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:38.864825 20534 server_base.cc:1034] running on GCE node
W20260806 14:58:38.864627 20547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:38.865144 20534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:38.865439 20534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:38.866603 20534 hybrid_clock.cc:648] HybridClock initialized: now 1786028318866588 us; error 43 us; skew 500 ppm
Aug 06 14:58:38 dist-test-slave-68xk krb5kdc[19575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028318, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:38.869073 20534 init.cc:377] Logged in from keytab as oryx/127.11.8.124@KRBTEST.COM (short username oryx)
I20260806 14:58:38.869526 20534 webserver.cc:460] Webserver started at http://127.11.8.124:36679/ using document root <none> and password file <none>
I20260806 14:58:38.869771 20534 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:38.869843 20534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:38.869997 20534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:38.871085 20534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-2/data/instance:
uuid: "ed19be708aac49f982e3e226c5180acd"
format_stamp: "Formatted at 2026-08-06 14:58:38 on dist-test-slave-68xk"
server_key: "e141a02e7a56fcbd310a4141d4b81a95"
server_key_iv: "f7c7b222e40ede5908c7163e4db9f8a6"
server_key_version: "encryptionkey@0"
I20260806 14:58:38.871457 20534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-2/wal/instance:
uuid: "ed19be708aac49f982e3e226c5180acd"
format_stamp: "Formatted at 2026-08-06 14:58:38 on dist-test-slave-68xk"
server_key: "e141a02e7a56fcbd310a4141d4b81a95"
server_key_iv: "f7c7b222e40ede5908c7163e4db9f8a6"
server_key_version: "encryptionkey@0"
I20260806 14:58:38.872756 20534 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 14:58:38.873474 20555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:38.873641 20534 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 14:58:38.873749 20534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-2/wal
uuid: "ed19be708aac49f982e3e226c5180acd"
format_stamp: "Formatted at 2026-08-06 14:58:38 on dist-test-slave-68xk"
server_key: "e141a02e7a56fcbd310a4141d4b81a95"
server_key_iv: "f7c7b222e40ede5908c7163e4db9f8a6"
server_key_version: "encryptionkey@0"
I20260806 14:58:38.873821 20534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:38.885041 20534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:38.886965 20534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:38.887125 20534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:38.891278 20534 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.124:34581
I20260806 14:58:38.891309 20634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.124:34581 every 8 connection(s)
I20260806 14:58:38.891672 20534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-2/data/info.pb
I20260806 14:58:38.892764 20636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:38.894304 20636 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } has no permanent_uuid. Determining permanent_uuid...
I20260806 14:58:38.898293 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 20534
Aug 06 14:58:38 dist-test-slave-68xk krb5kdc[19575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028318, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.124@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM
I20260806 14:58:38.904407 20638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.894989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.0.0.1:49792)
0806 14:58:38.895147 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:38.895188 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:38.895629 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:38.895913 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:38.895916 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:38.896148 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:38.896351 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.896355 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.897037 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.897038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.897812 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.897814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.898266 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.898268 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.898456 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.899190 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:38.899199 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:38.901099 (+ 1900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:38.902879 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.902886 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.902888 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.903208 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.903423 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.903424 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.903424 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.903580 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.903824 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:38.903830 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:38.903985 (+ 155us) client_negotiation.cc:769] Sending connection context
0806 14:58:38.904108 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 14:58:38.904250 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
Aug 06 14:58:38 dist-test-slave-68xk krb5kdc[19575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.8.124@KRBTEST.COM
I20260806 14:58:38.904810 19689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.895091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49792 (local address 127.11.8.126:34611)
0806 14:58:38.895181 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:38.895183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:38.895310 (+ 127us) server_negotiation.cc:407] Connection header received
0806 14:58:38.895741 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:38.895742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:38.895774 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:38.895849 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:38.896454 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.896949 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.897902 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.898130 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.898186 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.901189 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:38.901191 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:38.901191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:38.901198 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:38.902794 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.903293 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.903294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.903295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.903350 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.903652 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.903653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.903653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.903757 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:38.903809 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:38.904652 (+ 843us) server_negotiation.cc:299] Negotiation successful
0806 14:58:38.904738 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 14:58:38.905401 20636 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } has no permanent_uuid. Determining permanent_uuid...
I20260806 14:58:38.908201 20639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.898860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34076 (local address 127.11.8.124:34581)
0806 14:58:38.899098 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:38.899102 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:38.899124 (+ 22us) server_negotiation.cc:407] Connection header received
0806 14:58:38.899141 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:38.899142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:38.899157 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:38.899221 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:38.900016 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.900971 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.901992 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.902288 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.902376 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.905359 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:38.905363 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:38.905364 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:38.905371 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:38.907021 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.907401 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.907404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.907404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.907457 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.907654 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.907655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.907655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.907775 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:38.907941 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:38.908015 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 14:58:38.908087 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":97,"threads_started":1}
Aug 06 14:58:38 dist-test-slave-68xk krb5kdc[19575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028318, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.124@KRBTEST.COM for oryx/127.11.8.125@KRBTEST.COM
I20260806 14:58:38.912833 20638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.905673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:42909 (local address 127.0.0.1:42600)
0806 14:58:38.905715 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:38.905724 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:38.905755 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:38.905942 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:38.905943 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:38.906127 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:38.906175 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.906176 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.906931 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.906932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.907727 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.907729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.908106 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.908107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.908177 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.908877 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:38.908882 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:38.910157 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:38.911958 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.911959 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.911959 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.912130 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.912333 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.912334 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.912334 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.912389 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.912608 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:38.912608 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:38.912644 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 14:58:38.912696 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 14:58:38.912771 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 14:58:38.912846 20530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.905756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42600 (local address 127.11.8.125:42909)
0806 14:58:38.905821 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:38.905823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:38.905829 (+ 6us) server_negotiation.cc:407] Connection header received
0806 14:58:38.905861 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:38.905862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:38.905873 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:38.905939 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:38.906289 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.906848 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.907807 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.908033 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.908109 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.910255 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:38.910256 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:38.910256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:38.910263 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:38.911862 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.912204 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.912205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.912205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.912267 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.912444 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.912445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.912445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.912545 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:38.912595 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:38.912713 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 14:58:38.912787 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 14:58:38.913422 20636 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.124" port: 34581 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 14:58:38 dist-test-slave-68xk krb5kdc[19575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028318, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.124@KRBTEST.COM for oryx/127.11.8.124@KRBTEST.COM
I20260806 14:58:38.919970 20638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.913774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:34581 (local address 127.0.0.1:34082)
0806 14:58:38.913836 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:38.913847 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:38.913893 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:38.913988 (+ 95us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:38.913988 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:38.914101 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:38.914132 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.914133 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.914801 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.914802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.915286 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.915287 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.915623 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.915623 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.915662 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.916050 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:38.916054 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:38.917385 (+ 1331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:38.919132 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.919133 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.919133 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.919323 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.919508 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.919508 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.919508 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.919560 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.919745 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:38.919746 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:38.919779 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 14:58:38.919829 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 14:58:38.919904 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 14:58:38.920030 20639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.913829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34082 (local address 127.11.8.124:34581)
0806 14:58:38.913888 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:38.913890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:38.913898 (+ 8us) server_negotiation.cc:407] Connection header received
0806 14:58:38.913938 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:38.913939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:38.913948 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:38.913985 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:38.914192 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.914734 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.915360 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.915554 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.915609 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.917496 (+ 1887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:38.917498 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:38.917498 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:38.917505 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:38.919059 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.919397 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.919398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.919398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.919446 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.919598 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.919599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.919599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.919695 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:38.919739 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:38.919844 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 14:58:38.919948 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":40}
I20260806 14:58:38.921612 20636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd: Bootstrap starting.
I20260806 14:58:38.922588 20636 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:38.922881 20636 log.cc:824] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:38.923686 20636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd: No bootstrap required, opened a new log
I20260806 14:58:38.925269 20636 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 34581 } }
I20260806 14:58:38.925420 20636 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:38.925467 20636 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed19be708aac49f982e3e226c5180acd, State: Initialized, Role: FOLLOWER
I20260806 14:58:38.925625 20636 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 34581 } }
I20260806 14:58:38.925953 20636 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 14:58:38.925951 20641 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 34581 } } }
I20260806 14:58:38.926079 20641 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [sys.catalog]: This master's current role is: FOLLOWER
W20260806 14:58:38.927788 20652 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 14:58:38.927845 20652 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 14:58:38.930197 20534 master_runner.cc:428] Detected that this master ed19be708aac49f982e3e226c5180acd is joining an existing cluster
I20260806 14:58:38.930267 20534 master_runner.cc:432] Initiating AddMaster RPC to add 127.11.8.124:34581
I20260806 14:58:38.938527 19689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.931344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49794 (local address 127.11.8.126:34611)
0806 14:58:38.931392 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:38.931394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:38.931828 (+ 434us) server_negotiation.cc:407] Connection header received
0806 14:58:38.931926 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:38.931927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:38.931937 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:38.931967 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:38.932445 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.933212 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.934312 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.934648 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.934692 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.935806 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:38.935807 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:38.935808 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:38.935815 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:38.937353 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.937761 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.937762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.937762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.937815 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.938035 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.938036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.938036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.938137 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:38.938187 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:38.938346 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 14:58:38.938450 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 14:58:38.938582 20660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.931309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.0.0.1:49794)
0806 14:58:38.931758 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:38.931767 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:38.931854 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:38.932028 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:38.932028 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:38.932273 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:38.932348 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.932350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.933311 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.933312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.934209 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.934212 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.934779 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.934780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.934870 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.935257 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:38.935262 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:38.935705 (+ 443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:38.937435 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.937436 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.937437 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.937676 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.937888 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.937889 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.937890 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.937972 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.938204 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:38.938205 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:38.938251 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 14:58:38.938330 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 14:58:38.938463 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"mutex_wait_us":1,"thread_start_us":99,"threads_started":1}
Aug 06 14:58:38 dist-test-slave-68xk krb5kdc[19575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028317, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.126@KRBTEST.COM for oryx/127.11.8.124@KRBTEST.COM
I20260806 14:58:38.954118 20639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.931542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34094 (local address 127.11.8.124:34581)
0806 14:58:38.931586 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:38.931588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:38.931598 (+ 10us) server_negotiation.cc:407] Connection header received
0806 14:58:38.931618 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:38.931619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:38.931628 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:38.931689 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:38.932089 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.932803 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.933795 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.934070 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.934129 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.951061 (+ 16932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:38.951063 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:38.951063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:38.951070 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:38.952831 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.953321 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.953322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.953323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.953370 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.953563 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.953564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.953564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.953696 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:38.953739 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:38.953935 (+ 196us) server_negotiation.cc:299] Negotiation successful
0806 14:58:38.954022 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 14:58:38.954596 20661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.931305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:34581 (local address 127.0.0.1:34094)
0806 14:58:38.931482 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:38.931494 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:38.931560 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:38.931704 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:38.931705 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:38.931928 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:38.932009 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.932011 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.932883 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.932884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.933712 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.933714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.934236 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.934236 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.934290 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.934783 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:38.934788 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:38.950954 (+ 16166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:38.952926 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.952927 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.952927 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.953244 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.953439 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.953439 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.953439 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.953496 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.953783 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:38.953784 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:38.953827 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 14:58:38.954331 (+ 504us) client_negotiation.cc:240] Negotiation successful
0806 14:58:38.954464 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":82,"threads_started":1}
I20260806 14:58:38.954789 20530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.931581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42612 (local address 127.11.8.125:42909)
0806 14:58:38.931624 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:38.931626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:38.931868 (+ 242us) server_negotiation.cc:407] Connection header received
0806 14:58:38.931964 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:38.931965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:38.931978 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:38.932009 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:38.932456 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.933158 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.934264 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.934585 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.934620 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.951447 (+ 16827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:38.951449 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:38.951450 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:38.951459 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:38.953135 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.953700 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.953702 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.953702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.953768 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.953975 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.953976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.953976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.954102 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:38.954145 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:38.954572 (+ 427us) server_negotiation.cc:299] Negotiation successful
0806 14:58:38.954662 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 14:58:38.955000 20398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.940705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:34581 (local address 127.0.0.1:34100)
0806 14:58:38.940766 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:38.940777 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:38.940825 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:38.941174 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:38.941175 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:38.941311 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:38.941349 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.941350 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.942303 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.942304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.942879 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.942881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.943361 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.943362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.943413 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.943814 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:38.943819 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:38.945233 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:38.953821 (+ 8588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.953823 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.953823 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.954049 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.954325 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.954327 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.954327 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.954390 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.954697 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:38.954698 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:38.954743 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 14:58:38.954789 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 14:58:38.954896 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 14:58:38.955354 20664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.931661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:42909 (local address 127.0.0.1:42612)
0806 14:58:38.931807 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:38.931817 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:38.931892 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:38.932079 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:38.932080 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:38.932316 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:38.932369 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.932370 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.933260 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.933261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.934149 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.934151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.934718 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.934719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.934783 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.950867 (+ 16084us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:38.950873 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:38.951356 (+ 483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:38.953226 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.953227 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.953227 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.953619 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.953844 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.953844 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.953845 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.953904 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.954195 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:38.954196 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:38.954404 (+ 208us) client_negotiation.cc:769] Sending connection context
0806 14:58:38.955193 (+ 789us) client_negotiation.cc:240] Negotiation successful
0806 14:58:38.955287 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":40,"thread_start_us":75,"threads_started":1}
I20260806 14:58:38.956070 20665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.940885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34100 (local address 127.11.8.124:34581)
0806 14:58:38.941033 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:38.941035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:38.941044 (+ 9us) server_negotiation.cc:407] Connection header received
0806 14:58:38.941079 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:38.941080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:38.941090 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:38.941160 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:38.941431 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.942220 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.942953 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.943281 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.943354 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.945349 (+ 1995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:38.945351 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:38.945351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:38.945358 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:38.953715 (+ 8357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.954136 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.954136 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.954137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.954249 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.954486 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.954487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.954487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.954618 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:38.955788 (+ 1170us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:38.955826 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 14:58:38.955968 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":77,"threads_started":1}
I20260806 14:58:38.956534 19637 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.11.8.124:34581: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ed19be708aac49f982e3e226c5180acd"
member_type: NON_VOTER
last_known_addr {
host: "127.11.8.124"
port: 34581
}
attrs {
promote: true
}
}
dest_uuid: "d5ce68764a334405bdb27775b2321b6a"
cas_config_opid_index: 5
I20260806 14:58:38.956787 19637 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: NON_VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: true } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: NON_VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: true } }
I20260806 14:58:38.957154 20401 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ed19be708aac49f982e3e226c5180acd (127.11.8.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: NON_VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: true } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: NON_VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: true } } }
I20260806 14:58:38.957442 20596 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:38.957979 20401 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.11.8.124:34581
I20260806 14:58:38.958037 20401 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d5ce68764a334405bdb27775b2321b6a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: NON_VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: true } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: NON_VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: true } } }
I20260806 14:58:38.958083 20401 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:38.958359 20596 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [term 1 FOLLOWER]: Refusing update from remote peer d5ce68764a334405bdb27775b2321b6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 14:58:38.958657 20402 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: NON_VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:38.958727 20534 master.cc:537] Master@127.11.8.124:34581 shutting down...
W20260806 14:58:38.959185 19618 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260806 14:58:38.960963 19618 consensus_peers.cc:487] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a -> Peer ed19be708aac49f982e3e226c5180acd (127.11.8.124:34581): Couldn't send request to peer ed19be708aac49f982e3e226c5180acd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 14:58:38.962572 20530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.958545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42618 (local address 127.11.8.125:42909)
0806 14:58:38.958597 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:38.958599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:38.959061 (+ 462us) server_negotiation.cc:407] Connection header received
0806 14:58:38.959085 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:38.959086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:38.959108 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:38.959162 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:38.959420 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.959936 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.961421 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.962008 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.962093 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.962163 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:38.962460 (+ 297us) server_negotiation.cc:299] Negotiation successful
0806 14:58:38.962501 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 14:58:38.962738 20398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.957225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:42909 (local address 127.0.0.1:42618)
0806 14:58:38.958921 (+ 1696us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:38.958933 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:38.959003 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:38.959244 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:38.959245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:38.959247 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:38.959289 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.959290 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.960036 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.960037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.961342 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.961345 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.962214 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.962215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.962285 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.962304 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 14:58:38.962355 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 14:58:38.962394 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1678}
I20260806 14:58:38.962960 20475 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 1 LEARNER]: Refusing update from remote peer d5ce68764a334405bdb27775b2321b6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 14:58:38.963128 20402 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: NON_VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 14:58:38.963513 20475 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ed19be708aac49f982e3e226c5180acd (127.11.8.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: NON_VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: true } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: NON_VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: true } } }
I20260806 14:58:38.964540 20669 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d5ce68764a334405bdb27775b2321b6a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: NON_VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: true } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: NON_VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: true } } }
I20260806 14:58:38.964633 20512 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5ce68764a334405bdb27775b2321b6a. Latest consensus state: current_term: 1 leader_uuid: "d5ce68764a334405bdb27775b2321b6a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: NON_VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: true } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: NON_VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: true } } }
I20260806 14:58:38.964764 20512 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [sys.catalog]: This master's current role is: LEARNER
I20260806 14:58:38.964715 20669 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [sys.catalog]: This master's current role is: LEARNER
I20260806 14:58:38.965066 20534 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:58:38.965166 20534 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:38.965219 20534 tablet_replica.cc:330] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd: stopping tablet replica
W20260806 14:58:38.966710 19572 connection.cc:535] client connection to 127.11.8.124:34581 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 14:58:38.970862 20401 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a: attempting to promote NON_VOTER ef653e0dfa3841999c9d8f70b12f03e6 to VOTER
I20260806 14:58:38.971084 20401 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: NON_VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: true } }
I20260806 14:58:38.971477 20475 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 1 LEARNER]: Refusing update from remote peer d5ce68764a334405bdb27775b2321b6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 14:58:38.971544 20398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.971408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:34581 (local address 127.0.0.1:34108)
0806 14:58:38.971460 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:38.971505 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:34581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 14:58:38.971679 20399 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 14:58:38.971889 20398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.971766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:34581 (local address 127.0.0.1:34116)
0806 14:58:38.971827 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:38.971858 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:34581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 14:58:38.971990 19618 consensus_peers.cc:487] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a -> Peer ed19be708aac49f982e3e226c5180acd (127.11.8.124:34581): Couldn't send request to peer ed19be708aac49f982e3e226c5180acd. Status: Network error: Client connection negotiation failed: client connection to 127.11.8.124:34581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 14:58:38.972364 20402 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ef653e0dfa3841999c9d8f70b12f03e6 (127.11.8.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: NON_VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: true } } }
I20260806 14:58:38.973146 20399 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d5ce68764a334405bdb27775b2321b6a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: NON_VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: true } } }
I20260806 14:58:38.973250 20399 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:38.973229 20402 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d5ce68764a334405bdb27775b2321b6a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: NON_VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: true } } }
I20260806 14:58:38.973291 20402 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:38.973425 20475 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ef653e0dfa3841999c9d8f70b12f03e6 (127.11.8.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: NON_VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: true } } }
I20260806 14:58:38.974141 20669 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d5ce68764a334405bdb27775b2321b6a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: NON_VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: true } } }
I20260806 14:58:38.974232 20669 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 14:58:38.978209 20534 master.cc:552] Master@127.11.8.124:34581 shutdown complete.
I20260806 14:58:38.978258 20534 master_runner.cc:305] Clearing existing system tablet
I20260806 14:58:38.978394 20534 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:38.978468 20534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:38.979210 20534 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:38.979807 20671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:38.979990 20534 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:38.980053 20534 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 14:58:38.980079 20534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-2/wal
uuid: "ed19be708aac49f982e3e226c5180acd"
format_stamp: "Formatted at 2026-08-06 14:58:38 on dist-test-slave-68xk"
server_key: "e141a02e7a56fcbd310a4141d4b81a95"
server_key_iv: "f7c7b222e40ede5908c7163e4db9f8a6"
server_key_version: "encryptionkey@0"
I20260806 14:58:38.980895 20534 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:38.982326 20534 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 14:58:38.982400 20534 log.cc:1196] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 14:58:38.982694 20534 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd: Deleting consensus metadata
I20260806 14:58:38.982826 20534 master_runner.cc:315] Copying system tablet from 127.11.8.126:34611
I20260806 14:58:38.983449 20534 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd: tablet copy: Beginning tablet copy session from remote peer at address 127.11.8.126:34611
I20260806 14:58:38.989918 19689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.983861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49798 (local address 127.11.8.126:34611)
0806 14:58:38.983907 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:38.983909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:38.983976 (+ 67us) server_negotiation.cc:407] Connection header received
0806 14:58:38.984056 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:38.984057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:38.984065 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:38.984110 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:38.984480 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.985039 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.986108 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.986302 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.986361 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.987449 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:38.987451 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:38.987451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:38.987458 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:38.988884 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.989264 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.989265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.989265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.989315 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:38.989503 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:38.989504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:38.989504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:38.989600 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:38.989649 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:38.989800 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 14:58:38.989859 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 14:58:38.990010 20680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:38.983789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.0.0.1:49798)
0806 14:58:38.983909 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:38.983921 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:38.983995 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:38.984126 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:38.984127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:38.984328 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:38.984408 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.984410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.985126 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.985127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.986011 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:38.986014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:38.986381 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:38.986382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:38.986475 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:38.986955 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:38.986960 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:38.987361 (+ 401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:38.988967 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.988968 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.988968 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.989187 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.989380 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:38.989380 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:38.989381 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:38.989444 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:38.989661 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:38.989662 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:38.989708 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 14:58:38.989784 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 14:58:38.989887 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":76,"threads_started":1}
I20260806 14:58:38.990242 19662 tablet_copy_service.cc:140] P d5ce68764a334405bdb27775b2321b6a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ed19be708aac49f982e3e226c5180acd ({username='oryx', principal='oryx/127.11.8.124@KRBTEST.COM'} at 127.0.0.1:49798)
I20260806 14:58:38.990309 19662 tablet_copy_service.cc:161] P d5ce68764a334405bdb27775b2321b6a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ed19be708aac49f982e3e226c5180acd at {username='oryx', principal='oryx/127.11.8.124@KRBTEST.COM'} at 127.0.0.1:49798: session id = ed19be708aac49f982e3e226c5180acd-00000000000000000000000000000000
I20260806 14:58:38.990955 19662 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 14:58:38.991449 20534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 14:58:38.992539 20534 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd: tablet copy: Starting download of 0 data blocks...
I20260806 14:58:38.992715 20534 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd: tablet copy: Starting download of 1 WAL segments...
I20260806 14:58:38.994036 20534 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 14:58:38.994850 19662 tablet_copy_service.cc:342] P d5ce68764a334405bdb27775b2321b6a: Request end of tablet copy session ed19be708aac49f982e3e226c5180acd-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.11.8.124@KRBTEST.COM'} at 127.0.0.1:49798
I20260806 14:58:38.994915 19662 tablet_copy_service.cc:433] P d5ce68764a334405bdb27775b2321b6a: ending tablet copy session ed19be708aac49f982e3e226c5180acd-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ed19be708aac49f982e3e226c5180acd
I20260806 14:58:38.997299 20534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:38.998533 20681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:38.998571 20687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:38.998623 20682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:38.998876 20534 server_base.cc:1034] running on GCE node
I20260806 14:58:38.998975 20534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:38.999133 20534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:39.000253 20534 hybrid_clock.cc:648] HybridClock initialized: now 1786028319000234 us; error 34 us; skew 500 ppm
Aug 06 14:58:38 dist-test-slave-68xk krb5kdc[19575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028318, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:39.001705 20534 init.cc:377] Logged in from keytab as oryx/127.11.8.124@KRBTEST.COM (short username oryx)
I20260806 14:58:39.001955 20534 webserver.cc:460] Webserver started at http://127.11.8.124:45757/ using document root <none> and password file <none>
I20260806 14:58:39.002070 20534 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:39.002106 20534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:39.002835 20534 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 14:58:39.003407 20697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:39.003542 20534 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 14:58:39.003604 20534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-2/wal
uuid: "ed19be708aac49f982e3e226c5180acd"
format_stamp: "Formatted at 2026-08-06 14:58:38 on dist-test-slave-68xk"
server_key: "e141a02e7a56fcbd310a4141d4b81a95"
server_key_iv: "f7c7b222e40ede5908c7163e4db9f8a6"
server_key_version: "encryptionkey@0"
I20260806 14:58:39.003655 20534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:39.017825 20534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:39.019718 20534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:39.022284 20534 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.124:34581
I20260806 14:58:39.022320 20771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.124:34581 every 8 connection(s)
I20260806 14:58:39.022634 20534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/master-2/data/info.pb
I20260806 14:58:39.023597 20772 sys_catalog.cc:263] Verifying existing consensus state
I20260806 14:58:39.024133 20772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd: Bootstrap starting.
I20260806 14:58:39.027292 20772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 14:58:39.027520 20772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd: Bootstrap complete.
I20260806 14:58:39.027760 20772 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: NON_VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: true } }
I20260806 14:58:39.027858 20772 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ed19be708aac49f982e3e226c5180acd, State: Initialized, Role: LEARNER
I20260806 14:58:39.027900 20772 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: NON_VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: true } }
I20260806 14:58:39.028095 20772 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 14:58:39.028111 20775 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: NON_VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: true } } }
I20260806 14:58:39.028258 20775 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [sys.catalog]: This master's current role is: LEARNER
I20260806 14:58:39.029691 20787 catalog_manager.cc:1259] Loaded cluster ID: c461a7c22dcb49a6a9a5e229a9598c26
I20260806 14:58:39.029924 20787 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd: loading cluster ID for follower catalog manager: success
I20260806 14:58:39.030983 20787 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd: acquiring CA information for follower catalog manager: success
I20260806 14:58:39.031261 20787 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 14:58:39.074649 20788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.067792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34146 (local address 127.11.8.124:34581)
0806 14:58:39.067953 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:39.067955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:39.067962 (+ 7us) server_negotiation.cc:407] Connection header received
0806 14:58:39.067986 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:39.067986 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:39.067995 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:39.068080 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:39.068656 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.069333 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.070042 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.070268 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.070351 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.071783 (+ 1432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:39.071785 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:39.071785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:39.071795 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:39.073394 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.073815 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.073816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.073817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.073868 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.074064 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.074064 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.074065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.074162 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:39.074238 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:39.074365 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 14:58:39.074509 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":73,"threads_started":1}
I20260806 14:58:39.075237 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 19723
I20260806 14:58:39.088932 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:40721
--local_ip_for_outbound_sockets=127.11.8.65
--tserver_master_addrs=127.11.8.126:34611
--webserver_port=46109
--webserver_interface=127.11.8.65
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.65.keytab
--principal=oryx/127.11.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:36345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.8.126:34611,127.11.8.125:42909
--tserver_master_addrs=127.11.8.126:34611,127.11.8.125:42909,127.11.8.124:34581 with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:39.173300 20789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:39.173492 20789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:39.173516 20789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:39.174615 20789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:39.174649 20789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:39.174674 20789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 14:58:39.176225 20789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:36345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.65.keytab
--principal=oryx/127.11.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:40721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=46109
--tserver_master_addrs=127.11.8.126:34611,127.11.8.125:42909,127.11.8.124:34581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:39.176403 20789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:39.176580 20789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:39.181591 20796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:39.181597 20795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:39.181670 20789 server_base.cc:1034] running on GCE node
W20260806 14:58:39.181598 20799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:39.182071 20789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:39.182327 20789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:39.183497 20789 hybrid_clock.cc:648] HybridClock initialized: now 1786028319183498 us; error 40 us; skew 500 ppm
Aug 06 14:58:39 dist-test-slave-68xk krb5kdc[19575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028319, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:39.186203 20789 init.cc:377] Logged in from keytab as oryx/127.11.8.65@KRBTEST.COM (short username oryx)
I20260806 14:58:39.186702 20789 webserver.cc:460] Webserver started at http://127.11.8.65:46109/ using document root <none> and password file <none>
I20260806 14:58:39.186955 20789 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:39.187057 20789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:39.188428 20789 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:39.189044 20810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:39.189221 20789 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 14:58:39.189317 20789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "9e45adfc7114482c9bfd1cc0b6b75a96"
format_stamp: "Formatted at 2026-08-06 14:58:38 on dist-test-slave-68xk"
server_key: "2b7f956a6bf6230bb24d98ef4caf4f4a"
server_key_iv: "e03d38b1add7b04b2963a56a15d132ad"
server_key_version: "encryptionkey@0"
I20260806 14:58:39.189671 20789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:39.205008 20789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:39.207393 20789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:39.207563 20789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:39.207829 20789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:39.208164 20789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:39.208221 20789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:39.208256 20789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:39.208299 20789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:39.214272 20789 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:40721
I20260806 14:58:39.214318 20971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:40721 every 8 connection(s)
I20260806 14:58:39.214721 20789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
Aug 06 14:58:39 dist-test-slave-68xk krb5kdc[19575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028319, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM
Aug 06 14:58:39 dist-test-slave-68xk krb5kdc[19575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028319, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.125@KRBTEST.COM
Aug 06 14:58:39 dist-test-slave-68xk krb5kdc[19575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028319, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.124@KRBTEST.COM
I20260806 14:58:39.224902 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 20789
I20260806 14:58:39.225087 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 19891
I20260806 14:58:39.226661 19689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.216000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:48121 (local address 127.11.8.126:34611)
0806 14:58:39.216047 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:39.216050 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:39.216153 (+ 103us) server_negotiation.cc:407] Connection header received
0806 14:58:39.217226 (+ 1073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:39.217227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:39.217241 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:39.217276 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:39.218155 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.218923 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.220097 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.220382 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.220462 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.222851 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:39.222852 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:39.222853 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:39.222862 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:39.225473 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.225890 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.225891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.225891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.225943 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.226107 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.226108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.226108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.226227 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:39.226263 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:39.226452 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 14:58:39.226528 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 14:58:39.226737 20530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.215558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:56577 (local address 127.11.8.125:42909)
0806 14:58:39.216022 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:39.216025 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:39.216117 (+ 92us) server_negotiation.cc:407] Connection header received
0806 14:58:39.217224 (+ 1107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:39.217225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:39.217238 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:39.217286 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:39.218114 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.218923 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.219952 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.220227 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.220293 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.223332 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:39.223333 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:39.223333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:39.223341 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:39.224789 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.225362 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.225363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.225364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.225422 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.225724 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.225725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.225726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.225910 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:39.225965 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:39.226497 (+ 532us) server_negotiation.cc:299] Negotiation successful
0806 14:58:39.226598 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434}
I20260806 14:58:39.227454 20981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.215566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:42909 (local address 127.11.8.65:56577)
0806 14:58:39.216034 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:39.216045 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:39.217097 (+ 1052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:39.217365 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:39.217369 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:39.217825 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:39.218024 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.218029 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.219024 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.219025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.219856 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.219858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.220311 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.220312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.220448 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.220963 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:39.220974 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:39.223232 (+ 2258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:39.224890 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:39.224897 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:39.224900 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:39.225268 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:39.225491 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:39.225492 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:39.225492 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:39.225636 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:39.226021 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:39.226029 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:39.226260 (+ 231us) client_negotiation.cc:769] Sending connection context
0806 14:58:39.226882 (+ 622us) client_negotiation.cc:240] Negotiation successful
0806 14:58:39.227110 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"mutex_wait_us":42,"thread_start_us":145,"threads_started":1}
I20260806 14:58:39.227455 20980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.215889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.11.8.65:48121)
0806 14:58:39.216082 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:39.216091 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:39.217106 (+ 1015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:39.217364 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:39.217369 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:39.217813 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:39.218025 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.218031 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.218990 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.218990 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.220010 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.220013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.220463 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.220464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.220529 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.221230 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:39.221235 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:39.222752 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:39.225586 (+ 2834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:39.225588 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:39.225588 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:39.225806 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:39.226003 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:39.226004 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:39.226004 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:39.226052 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:39.226300 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:39.226301 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:39.226339 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 14:58:39.227079 (+ 740us) client_negotiation.cc:240] Negotiation successful
0806 14:58:39.227184 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":94,"thread_start_us":74,"threads_started":1}
I20260806 14:58:39.230561 20976 heartbeater.cc:347] Connected to a master server at 127.11.8.125:42909
I20260806 14:58:39.230674 20976 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:39.230888 20976 heartbeater.cc:511] Master 127.11.8.125:42909 requested a full tablet report, sending...
I20260806 14:58:39.231480 20463 ts_manager.cc:194] Registered new tserver with Master: 9e45adfc7114482c9bfd1cc0b6b75a96 (127.11.8.65:40721)
I20260806 14:58:39.231875 20788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.215587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:47071 (local address 127.11.8.124:34581)
0806 14:58:39.215644 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:39.215647 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:39.215920 (+ 273us) server_negotiation.cc:407] Connection header received
0806 14:58:39.217237 (+ 1317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:39.217238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:39.217254 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:39.217284 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:39.218273 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.218836 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.220121 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.220394 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.220436 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.223876 (+ 3440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:39.223877 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:39.223877 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:39.223885 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:39.225776 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.230981 (+ 5205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.230983 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.230983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.231047 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.231269 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.231270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.231270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.231397 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:39.231441 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:39.231682 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 14:58:39.231783 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 14:58:39.232311 20979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.215565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:34581 (local address 127.11.8.65:47071)
0806 14:58:39.215768 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:39.215811 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:39.217094 (+ 1283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:39.217364 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:39.217369 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:39.217804 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:39.218181 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.218183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.218944 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.218945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.220013 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.220015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.220517 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.220517 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.220590 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.221323 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:39.221328 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:39.223780 (+ 2452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:39.230651 (+ 6871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:39.230652 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:39.230653 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:39.230876 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:39.231127 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:39.231127 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:39.231128 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:39.231190 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:39.231493 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:39.231495 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:39.231554 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 14:58:39.231980 (+ 426us) client_negotiation.cc:240] Negotiation successful
0806 14:58:39.232090 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":104,"thread_start_us":120,"threads_started":1}
I20260806 14:58:39.232710 20972 heartbeater.cc:347] Connected to a master server at 127.11.8.124:34581
I20260806 14:58:39.232753 20972 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:39.232813 20972 heartbeater.cc:511] Master 127.11.8.124:34581 requested a full tablet report, sending...
I20260806 14:58:39.233170 20712 ts_manager.cc:194] Registered new tserver with Master: 9e45adfc7114482c9bfd1cc0b6b75a96 (127.11.8.65:40721)
I20260806 14:58:39.238978 20974 heartbeater.cc:347] Connected to a master server at 127.11.8.126:34611
I20260806 14:58:39.239089 20974 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:39.239181 20974 heartbeater.cc:511] Master 127.11.8.126:34611 requested a full tablet report, sending...
I20260806 14:58:39.239424 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:46791
--local_ip_for_outbound_sockets=127.11.8.66
--tserver_master_addrs=127.11.8.126:34611
--webserver_port=38151
--webserver_interface=127.11.8.66
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.66.keytab
--principal=oryx/127.11.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:36345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.8.126:34611,127.11.8.125:42909
--tserver_master_addrs=127.11.8.126:34611,127.11.8.125:42909,127.11.8.124:34581 with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 14:58:39.239604 19637 ts_manager.cc:194] Re-registered known tserver with Master: 9e45adfc7114482c9bfd1cc0b6b75a96 (127.11.8.65:40721)
I20260806 14:58:39.240001 19637 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.8.65@KRBTEST.COM'} at 127.11.8.65:48121
W20260806 14:58:39.323732 20982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:39.323930 20982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:39.323951 20982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:39.325119 20982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:39.325162 20982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:39.325188 20982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 14:58:39.326630 20982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:36345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.66.keytab
--principal=oryx/127.11.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:46791
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=38151
--tserver_master_addrs=127.11.8.126:34611,127.11.8.125:42909,127.11.8.124:34581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:39.326828 20982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:39.326992 20982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:39.332178 20987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:39.332156 20988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:39.332178 20990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:39.332463 20982 server_base.cc:1034] running on GCE node
I20260806 14:58:39.332712 20982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:39.332950 20982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:39.334141 20982 hybrid_clock.cc:648] HybridClock initialized: now 1786028319334117 us; error 36 us; skew 500 ppm
Aug 06 14:58:39 dist-test-slave-68xk krb5kdc[19575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028319, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:39.336694 20982 init.cc:377] Logged in from keytab as oryx/127.11.8.66@KRBTEST.COM (short username oryx)
I20260806 14:58:39.337122 20982 webserver.cc:460] Webserver started at http://127.11.8.66:38151/ using document root <none> and password file <none>
I20260806 14:58:39.337424 20982 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:39.337486 20982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:39.338999 20982 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 14:58:39.339852 20999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:39.340042 20982 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 14:58:39.340170 20982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "cd4e5e034bb647b2adf0bbf9f4850843"
format_stamp: "Formatted at 2026-08-06 14:58:38 on dist-test-slave-68xk"
server_key: "d3a842c99ec0ed2c4edd38261e12cb73"
server_key_iv: "55798dc5e1a9e3d1be762676adfdfab4"
server_key_version: "encryptionkey@0"
I20260806 14:58:39.340548 20982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:39.365293 20982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:39.367316 20982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:39.367491 20982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:39.367740 20982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:39.368208 20982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:39.368292 20982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:39.368336 20982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:39.368358 20982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:39.374974 20982 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:46791
I20260806 14:58:39.375092 21160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:46791 every 8 connection(s)
I20260806 14:58:39.375522 20982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
Aug 06 14:58:39 dist-test-slave-68xk krb5kdc[19575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028319, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.125@KRBTEST.COM
Aug 06 14:58:39 dist-test-slave-68xk krb5kdc[19575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028319, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM
Aug 06 14:58:39 dist-test-slave-68xk krb5kdc[19575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028319, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.124@KRBTEST.COM
I20260806 14:58:39.385363 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 20982
I20260806 14:58:39.385540 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 20076
I20260806 14:58:39.388935 19689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.376327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:50481 (local address 127.11.8.126:34611)
0806 14:58:39.376396 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:39.376397 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:39.376811 (+ 414us) server_negotiation.cc:407] Connection header received
0806 14:58:39.378033 (+ 1222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:39.378034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:39.378047 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:39.378098 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:39.378881 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.379664 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.380559 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.380849 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.380917 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.384068 (+ 3151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:39.384070 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:39.384070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:39.384078 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:39.386915 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.387549 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.387551 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.387551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.387621 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.387966 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.387968 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.387968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.388155 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:39.388210 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:39.388683 (+ 473us) server_negotiation.cc:299] Negotiation successful
0806 14:58:39.388782 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 14:58:39.389679 21170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.376281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.11.8.66:50481)
0806 14:58:39.376714 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:39.376725 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:39.377770 (+ 1045us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:39.378153 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:39.378154 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:39.378560 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:39.378759 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.378766 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.379756 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.379757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.380452 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.380454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.380966 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.380968 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.381117 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.381763 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:39.381773 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:39.383969 (+ 2196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:39.387037 (+ 3068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:39.387046 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:39.387049 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:39.387443 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:39.387711 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:39.387712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:39.387713 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:39.387869 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:39.388269 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:39.388277 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:39.388485 (+ 208us) client_negotiation.cc:769] Sending connection context
0806 14:58:39.389081 (+ 596us) client_negotiation.cc:240] Negotiation successful
0806 14:58:39.389333 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"mutex_wait_us":26,"thread_start_us":101,"threads_started":1}
I20260806 14:58:39.390600 21165 heartbeater.cc:347] Connected to a master server at 127.11.8.126:34611
I20260806 14:58:39.390691 21165 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:39.390833 21165 heartbeater.cc:511] Master 127.11.8.126:34611 requested a full tablet report, sending...
I20260806 14:58:39.391182 19637 ts_manager.cc:194] Re-registered known tserver with Master: cd4e5e034bb647b2adf0bbf9f4850843 (127.11.8.66:46791)
I20260806 14:58:39.391844 19637 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.8.66@KRBTEST.COM'} at 127.11.8.66:50481
I20260806 14:58:39.393121 21169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.376596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:42909 (local address 127.11.8.66:41829)
0806 14:58:39.376760 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:39.376770 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:39.377750 (+ 980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:39.378046 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:39.378056 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:39.378556 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:39.378759 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.378766 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.379711 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.379712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.380611 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.380614 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.381385 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.381386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.381465 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.381951 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:39.381957 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:39.384541 (+ 2584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:39.391984 (+ 7443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:39.391985 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:39.391986 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:39.392264 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:39.392482 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:39.392483 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:39.392483 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:39.392546 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:39.392801 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:39.392802 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:39.392839 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 14:58:39.392895 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 14:58:39.392991 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":35,"thread_start_us":68,"threads_started":1}
I20260806 14:58:39.393218 20530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.376666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:41829 (local address 127.11.8.125:42909)
0806 14:58:39.376746 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:39.376748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:39.376818 (+ 70us) server_negotiation.cc:407] Connection header received
0806 14:58:39.377877 (+ 1059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:39.377878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:39.377892 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:39.377926 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:39.378879 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.379630 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.381106 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.381302 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.381384 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.384672 (+ 3288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:39.384674 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:39.384675 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:39.384684 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:39.391340 (+ 6656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.392356 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.392357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.392357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.392417 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.392604 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.392605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.392605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.392726 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:39.392763 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:39.392975 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 14:58:39.393079 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 14:58:39.393569 21166 heartbeater.cc:347] Connected to a master server at 127.11.8.125:42909
I20260806 14:58:39.393615 21166 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:39.393679 21166 heartbeater.cc:511] Master 127.11.8.125:42909 requested a full tablet report, sending...
I20260806 14:58:39.394063 20463 ts_manager.cc:194] Registered new tserver with Master: cd4e5e034bb647b2adf0bbf9f4850843 (127.11.8.66:46791)
I20260806 14:58:39.396533 20788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.376281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:56507 (local address 127.11.8.124:34581)
0806 14:58:39.376393 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:39.376396 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:39.376774 (+ 378us) server_negotiation.cc:407] Connection header received
0806 14:58:39.377880 (+ 1106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:39.377881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:39.377895 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:39.377954 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:39.378859 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.379374 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.380699 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.380996 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.381059 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.384673 (+ 3614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:39.384675 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:39.384675 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:39.384685 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:39.395326 (+ 10641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.395701 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.395703 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.395703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.395775 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.395974 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.395976 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.395976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.396108 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:39.396202 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:39.396355 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 14:58:39.396445 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 14:58:39.396554 21168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.376281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:34581 (local address 127.11.8.66:56507)
0806 14:58:39.376578 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:39.376624 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:39.377750 (+ 1126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:39.378046 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:39.378050 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:39.378524 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:39.378759 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.378764 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.379490 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.379491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.380614 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.380616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.381088 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.381089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.381146 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.382179 (+ 1033us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:39.382184 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:39.384583 (+ 2399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:39.395424 (+ 10841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:39.395425 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:39.395426 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:39.395613 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:39.395846 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:39.395847 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:39.395847 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:39.395908 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:39.396196 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:39.396197 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:39.396242 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 14:58:39.396330 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 14:58:39.396444 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":7,"thread_start_us":99,"threads_started":1}
I20260806 14:58:39.396862 21162 heartbeater.cc:347] Connected to a master server at 127.11.8.124:34581
I20260806 14:58:39.396912 21162 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:39.396963 21162 heartbeater.cc:511] Master 127.11.8.124:34581 requested a full tablet report, sending...
I20260806 14:58:39.397244 20712 ts_manager.cc:194] Registered new tserver with Master: cd4e5e034bb647b2adf0bbf9f4850843 (127.11.8.66:46791)
I20260806 14:58:39.398557 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:34831
--local_ip_for_outbound_sockets=127.11.8.67
--tserver_master_addrs=127.11.8.126:34611
--webserver_port=34807
--webserver_interface=127.11.8.67
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.67.keytab
--principal=oryx/127.11.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:36345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.11.8.126:34611,127.11.8.125:42909
--tserver_master_addrs=127.11.8.126:34611,127.11.8.125:42909,127.11.8.124:34581 with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:39.481141 21171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:39.481333 21171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:39.481356 21171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:39.482537 21171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:39.482578 21171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:39.482609 21171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 14:58:39.484053 21171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:36345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.67.keytab
--principal=oryx/127.11.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:34831
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=34807
--tserver_master_addrs=127.11.8.126:34611,127.11.8.125:42909,127.11.8.124:34581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:39.484273 21171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:39.484515 21171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:39.489533 21182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:39.489610 21179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:39.489751 21171 server_base.cc:1034] running on GCE node
W20260806 14:58:39.489784 21180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:39.489979 21171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:39.490209 21171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:39.491369 21171 hybrid_clock.cc:648] HybridClock initialized: now 1786028319491345 us; error 59 us; skew 500 ppm
Aug 06 14:58:39 dist-test-slave-68xk krb5kdc[19575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028319, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:39.494165 21171 init.cc:377] Logged in from keytab as oryx/127.11.8.67@KRBTEST.COM (short username oryx)
I20260806 14:58:39.494634 21171 webserver.cc:460] Webserver started at http://127.11.8.67:34807/ using document root <none> and password file <none>
I20260806 14:58:39.494820 21171 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:39.494872 21171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:39.496138 21171 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:39.496713 21191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:39.496891 21171 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 14:58:39.496980 21171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "7887589cb7404641939ec7d93266fcca"
format_stamp: "Formatted at 2026-08-06 14:58:38 on dist-test-slave-68xk"
server_key: "91c7fc0564b815216df850b095aae6f1"
server_key_iv: "d759ecafd136cda2b19182947074bf76"
server_key_version: "encryptionkey@0"
I20260806 14:58:39.497285 21171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:39.523672 21171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:39.526146 21171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:39.526264 21171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:39.526517 21171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:39.526803 21171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:39.526834 21171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:39.526856 21171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:39.526868 21171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:39.533258 21171 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:34831
I20260806 14:58:39.533370 21357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:34831 every 8 connection(s)
I20260806 14:58:39.533702 21171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
Aug 06 14:58:39 dist-test-slave-68xk krb5kdc[19575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028319, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM
Aug 06 14:58:39 dist-test-slave-68xk krb5kdc[19575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028319, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.125@KRBTEST.COM
Aug 06 14:58:39 dist-test-slave-68xk krb5kdc[19575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028319, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.124@KRBTEST.COM
I20260806 14:58:39.543991 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 21171
I20260806 14:58:39.545995 19689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.535014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:60167 (local address 127.11.8.126:34611)
0806 14:58:39.535057 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:39.535059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:39.535068 (+ 9us) server_negotiation.cc:407] Connection header received
0806 14:58:39.536058 (+ 990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:39.536060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:39.536075 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:39.536113 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:39.536904 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.537779 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.539493 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.539679 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.539763 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.542049 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:39.542051 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:39.542051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:39.542060 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:39.544016 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.544511 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.544512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.544513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.544575 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.545191 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.545192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.545192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.545355 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:39.545401 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:39.545799 (+ 398us) server_negotiation.cc:299] Negotiation successful
0806 14:58:39.545895 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 14:58:39.546082 20530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.534801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:54905 (local address 127.11.8.125:42909)
0806 14:58:39.534857 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:39.534859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:39.534869 (+ 10us) server_negotiation.cc:407] Connection header received
0806 14:58:39.536058 (+ 1189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:39.536060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:39.536075 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:39.536113 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:39.536902 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.537733 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.539066 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.539367 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.539434 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.542525 (+ 3091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:39.542527 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:39.542528 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:39.542537 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:39.544654 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.545194 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.545196 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.545196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.545268 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.545524 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.545525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.545526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.545652 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:39.545708 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:39.545909 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 14:58:39.545982 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 14:58:39.546614 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.534822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.11.8.67:60167)
0806 14:58:39.534999 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:39.535009 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:39.535916 (+ 907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:39.536220 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:39.536224 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:39.536639 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:39.536777 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.536780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.537879 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.537880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.538929 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.538930 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.539763 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.539764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.539836 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.540377 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:39.540383 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:39.541942 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:39.544114 (+ 2172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:39.544121 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:39.544123 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:39.544413 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:39.544667 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:39.544667 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:39.544668 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:39.544800 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:39.545450 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:39.545458 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:39.545632 (+ 174us) client_negotiation.cc:769] Sending connection context
0806 14:58:39.546120 (+ 488us) client_negotiation.cc:240] Negotiation successful
0806 14:58:39.546338 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":33,"thread_start_us":66,"threads_started":1}
I20260806 14:58:39.546613 21368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.534495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:42909 (local address 127.11.8.67:54905)
0806 14:58:39.534686 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:39.534731 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:39.535916 (+ 1185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:39.536221 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:39.536224 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:39.536642 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:39.536777 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.536780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.537850 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.537851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.538926 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.538929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.539446 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.539447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.539545 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.540247 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:39.540252 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:39.542444 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:39.544895 (+ 2451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:39.544896 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:39.544896 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:39.545077 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:39.545342 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:39.545343 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:39.545343 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:39.545449 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:39.545743 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:39.545744 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:39.545781 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 14:58:39.546178 (+ 397us) client_negotiation.cc:240] Negotiation successful
0806 14:58:39.546389 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":175,"thread_start_us":118,"threads_started":1}
I20260806 14:58:39.547150 21365 heartbeater.cc:347] Connected to a master server at 127.11.8.125:42909
I20260806 14:58:39.547235 21365 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:39.547379 21365 heartbeater.cc:511] Master 127.11.8.125:42909 requested a full tablet report, sending...
I20260806 14:58:39.547792 20463 ts_manager.cc:194] Registered new tserver with Master: 7887589cb7404641939ec7d93266fcca (127.11.8.67:34831)
I20260806 14:58:39.548276 21360 heartbeater.cc:347] Connected to a master server at 127.11.8.126:34611
I20260806 14:58:39.548331 21360 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:39.548400 21360 heartbeater.cc:511] Master 127.11.8.126:34611 requested a full tablet report, sending...
I20260806 14:58:39.548674 19637 ts_manager.cc:194] Re-registered known tserver with Master: 7887589cb7404641939ec7d93266fcca (127.11.8.67:34831)
I20260806 14:58:39.549070 20788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.534536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:50561 (local address 127.11.8.124:34581)
0806 14:58:39.534594 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:39.534597 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:39.534825 (+ 228us) server_negotiation.cc:407] Connection header received
0806 14:58:39.536037 (+ 1212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:39.536038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:39.536052 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:39.536143 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:39.536869 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.537405 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.538986 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.539292 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.539362 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.543140 (+ 3778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:39.543141 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:39.543141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:39.543149 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:39.545208 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.545699 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.545700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.545701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.545785 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.546020 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.546021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.546021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.546176 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:39.546247 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:39.548907 (+ 2660us) server_negotiation.cc:299] Negotiation successful
0806 14:58:39.548985 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 14:58:39.549094 21367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.534496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:34581 (local address 127.11.8.67:50561)
0806 14:58:39.534686 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:39.534729 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:39.535914 (+ 1185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:39.536219 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:39.536223 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:39.536575 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:39.536775 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.536779 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.537534 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.537535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.538894 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.538897 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.539379 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.539379 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.539545 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.540176 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:39.540207 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:39.543050 (+ 2843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:39.545311 (+ 2261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:39.545312 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:39.545313 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:39.545600 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:39.545871 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:39.545872 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:39.545873 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:39.545943 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:39.548729 (+ 2786us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:39.548730 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:39.548777 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 14:58:39.548851 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 14:58:39.548955 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":1,"thread_start_us":116,"threads_started":1}
I20260806 14:58:39.549211 19637 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.8.67@KRBTEST.COM'} at 127.11.8.67:60167
I20260806 14:58:39.549415 21358 heartbeater.cc:347] Connected to a master server at 127.11.8.124:34581
I20260806 14:58:39.549471 21358 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:39.549539 21358 heartbeater.cc:511] Master 127.11.8.124:34581 requested a full tablet report, sending...
I20260806 14:58:39.549845 20712 ts_manager.cc:194] Registered new tserver with Master: 7887589cb7404641939ec7d93266fcca (127.11.8.67:34831)
I20260806 14:58:39.555928 21376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.545345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34162 (local address 127.11.8.124:34581)
0806 14:58:39.545691 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:39.545693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:39.545905 (+ 212us) server_negotiation.cc:407] Connection header received
0806 14:58:39.545928 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:39.545929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:39.545941 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:39.546006 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:39.546948 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.547843 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.549051 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.549443 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.549517 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.551360 (+ 1843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:39.551361 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:39.551361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:39.551369 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:39.554058 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.554478 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.554479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.554479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.554544 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.554762 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.554763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.554764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.554917 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:39.555501 (+ 584us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:39.555681 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 14:58:39.555792 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":61,"threads_started":1}
I20260806 14:58:39.559144 19689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.550616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49802 (local address 127.11.8.126:34611)
0806 14:58:39.550651 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:39.550653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:39.550866 (+ 213us) server_negotiation.cc:407] Connection header received
0806 14:58:39.550936 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:39.550937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:39.550947 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:39.550999 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:39.551826 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.552693 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.554321 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.554673 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.554776 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.556010 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:39.556011 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:39.556011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:39.556019 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:39.557639 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.557984 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.557985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.557985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.558044 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.558244 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.558245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.558245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.558379 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:39.558585 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:39.558920 (+ 335us) server_negotiation.cc:299] Negotiation successful
0806 14:58:39.559016 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 14:58:39.559719 20530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.551009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42626 (local address 127.11.8.125:42909)
0806 14:58:39.551048 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:39.551050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:39.551295 (+ 245us) server_negotiation.cc:407] Connection header received
0806 14:58:39.551335 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:39.551336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:39.551349 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:39.551405 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:39.551961 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.552750 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.553896 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.554238 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.554329 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.556825 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:39.556826 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:39.556826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:39.556832 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:39.558253 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.558741 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.558742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.558742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.558809 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.559041 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.559042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.559043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.559148 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:39.559224 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:39.559536 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 14:58:39.559633 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 14:58:39.560784 19637 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49802:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 14:58:39.562541 21380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.558799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:34581 (local address 127.0.0.1:34176)
0806 14:58:39.559354 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:39.559365 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:39.559447 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:39.559672 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:39.559673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:39.559675 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:39.559721 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.559723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.560736 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.560737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.561558 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.561560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.562257 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.562258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.562326 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.562331 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 14:58:39.562398 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 14:58:39.562467 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":71,"threads_started":1}
I20260806 14:58:39.562533 21376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.558847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34176 (local address 127.11.8.124:34581)
0806 14:58:39.559301 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:39.559303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:39.559419 (+ 116us) server_negotiation.cc:407] Connection header received
0806 14:58:39.559532 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:39.559533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:39.559557 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:39.559603 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:39.559811 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.560631 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.561700 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.562154 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.562256 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.562334 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:39.562404 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 14:58:39.562465 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428}
W20260806 14:58:39.562639 19637 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 14:58:39.563026 20738 raft_consensus.cc:1203] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260806 14:58:39.563268 21383 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5ce68764a334405bdb27775b2321b6a. Latest consensus state: current_term: 1 leader_uuid: "d5ce68764a334405bdb27775b2321b6a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: NON_VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: true } } }
I20260806 14:58:39.563395 21383 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [sys.catalog]: This master's current role is: LEARNER
I20260806 14:58:39.563673 20401 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d5ce68764a334405bdb27775b2321b6a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: NON_VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: true } } }
I20260806 14:58:39.563771 20401 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:39.565285 20401 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a: attempting to promote NON_VOTER ed19be708aac49f982e3e226c5180acd to VOTER
I20260806 14:58:39.565407 20401 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: false } }
I20260806 14:58:39.565876 20475 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 1 FOLLOWER]: Refusing update from remote peer d5ce68764a334405bdb27775b2321b6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260806 14:58:39.565901 20737 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [term 1 LEARNER]: Refusing update from remote peer d5ce68764a334405bdb27775b2321b6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260806 14:58:39.566073 21387 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 14:58:39.566154 20401 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260806 14:58:39.566944 21386 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 7 to 10, ed19be708aac49f982e3e226c5180acd (127.11.8.124) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: false } } }
I20260806 14:58:39.567013 20475 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 7 to 10, ed19be708aac49f982e3e226c5180acd (127.11.8.124) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: false } } }
I20260806 14:58:39.567122 20737 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 7 to 10, ed19be708aac49f982e3e226c5180acd (127.11.8.124) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: false } } }
I20260806 14:58:39.568109 21386 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d5ce68764a334405bdb27775b2321b6a" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: false } } }
I20260806 14:58:39.568112 20401 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d5ce68764a334405bdb27775b2321b6a" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: false } } }
I20260806 14:58:39.568207 21386 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:39.568226 20401 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:39.568288 21386 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d5ce68764a334405bdb27775b2321b6a" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: false } } }
I20260806 14:58:39.568348 21386 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:39.568295 21383 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d5ce68764a334405bdb27775b2321b6a" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: false } } }
I20260806 14:58:39.568384 21383 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 14:58:39.569041 21398 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d5ce68764a334405bdb27775b2321b6a" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: false } } }
I20260806 14:58:39.569185 21398 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 14:58:39.573419 21413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.569090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:34831 (local address 127.0.0.1:56828)
0806 14:58:39.569345 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:39.569356 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:39.569430 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:39.569760 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:39.569761 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:39.569763 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:39.569815 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.569817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.571149 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.571150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.572222 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.572224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.573146 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.573148 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.573218 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.573224 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 14:58:39.573279 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 14:58:39.573323 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":4,"thread_start_us":91,"threads_started":1}
I20260806 14:58:39.573586 21416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.569623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45418 (local address 127.11.8.65:40721)
0806 14:58:39.569960 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:39.569965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:39.569975 (+ 10us) server_negotiation.cc:407] Connection header received
0806 14:58:39.570014 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:39.570016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:39.570032 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:39.570148 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:39.570237 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.570978 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.572315 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.573180 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.573283 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.573363 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:39.573426 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 14:58:39.573478 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":94,"threads_started":1}
I20260806 14:58:39.573647 21380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.569183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:40721 (local address 127.0.0.1:45418)
0806 14:58:39.569338 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:39.569348 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:39.569413 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:39.570146 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:39.570146 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:39.570148 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:39.570182 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.570184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.571060 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.571061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.572212 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.572215 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.573280 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.573281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.573348 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.573351 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 14:58:39.573408 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 14:58:39.573453 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":30,"thread_start_us":110,"threads_started":1}
I20260806 14:58:39.573873 21408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.569000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:46791 (local address 127.0.0.1:53160)
0806 14:58:39.569287 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:39.569300 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:39.569424 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:39.569864 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:39.569865 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:39.569867 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:39.569921 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.569922 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.571558 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.571559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.572411 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.572414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.573331 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.573332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.573400 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.573404 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 14:58:39.573434 (+ 30us) client_negotiation.cc:240] Negotiation successful
0806 14:58:39.573462 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264}
I20260806 14:58:39.573894 21415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.569300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56828 (local address 127.11.8.67:34831)
0806 14:58:39.569562 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:39.569567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:39.569578 (+ 11us) server_negotiation.cc:407] Connection header received
0806 14:58:39.569624 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:39.569627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:39.569644 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:39.569765 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:39.569893 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.571077 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.572316 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.573028 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.573119 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.573192 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:39.573609 (+ 417us) server_negotiation.cc:299] Negotiation successful
0806 14:58:39.573665 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":73,"threads_started":1}
I20260806 14:58:39.574136 21411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.569126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53160 (local address 127.11.8.66:46791)
0806 14:58:39.569657 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:39.569661 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:39.569672 (+ 11us) server_negotiation.cc:407] Connection header received
0806 14:58:39.569710 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:39.569712 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:39.569728 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:39.569846 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:39.570012 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.571465 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.572509 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.573238 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.573625 (+ 387us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.573835 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:39.573864 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 14:58:39.573992 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":92,"threads_started":1}
I20260806 14:58:39.574494 20878 tablet_service.cc:1467] Processing CreateTablet for tablet 78cee5e85c3f41fbb91ddbf28e761dd6 (DEFAULT_TABLE table=test-table [id=4eac8ae65a5d47f0a2fe9bed6886adcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:39.574654 21265 tablet_service.cc:1467] Processing CreateTablet for tablet 78cee5e85c3f41fbb91ddbf28e761dd6 (DEFAULT_TABLE table=test-table [id=4eac8ae65a5d47f0a2fe9bed6886adcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:39.574926 20878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78cee5e85c3f41fbb91ddbf28e761dd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:39.574998 21265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78cee5e85c3f41fbb91ddbf28e761dd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:39.574739 21070 tablet_service.cc:1467] Processing CreateTablet for tablet 78cee5e85c3f41fbb91ddbf28e761dd6 (DEFAULT_TABLE table=test-table [id=4eac8ae65a5d47f0a2fe9bed6886adcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:39.575433 21070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78cee5e85c3f41fbb91ddbf28e761dd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:39.578138 21417 tablet_bootstrap.cc:492] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 9e45adfc7114482c9bfd1cc0b6b75a96: Bootstrap starting.
I20260806 14:58:39.578663 21419 tablet_bootstrap.cc:492] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca: Bootstrap starting.
I20260806 14:58:39.579325 21417 tablet_bootstrap.cc:654] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 9e45adfc7114482c9bfd1cc0b6b75a96: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:39.579424 21419 tablet_bootstrap.cc:654] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:39.579726 21417 log.cc:824] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 9e45adfc7114482c9bfd1cc0b6b75a96: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:39.579762 21419 log.cc:824] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:39.580113 21420 tablet_bootstrap.cc:492] T 78cee5e85c3f41fbb91ddbf28e761dd6 P cd4e5e034bb647b2adf0bbf9f4850843: Bootstrap starting.
I20260806 14:58:39.580616 21419 tablet_bootstrap.cc:492] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca: No bootstrap required, opened a new log
I20260806 14:58:39.580616 21417 tablet_bootstrap.cc:492] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 9e45adfc7114482c9bfd1cc0b6b75a96: No bootstrap required, opened a new log
I20260806 14:58:39.580699 21419 ts_tablet_manager.cc:1397] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 14:58:39.580699 21417 ts_tablet_manager.cc:1397] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 9e45adfc7114482c9bfd1cc0b6b75a96: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 14:58:39.582885 21420 tablet_bootstrap.cc:654] T 78cee5e85c3f41fbb91ddbf28e761dd6 P cd4e5e034bb647b2adf0bbf9f4850843: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:39.583249 21420 log.cc:824] T 78cee5e85c3f41fbb91ddbf28e761dd6 P cd4e5e034bb647b2adf0bbf9f4850843: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:39.583519 21419 raft_consensus.cc:348] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd4e5e034bb647b2adf0bbf9f4850843" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 46791 } } peers { permanent_uuid: "7887589cb7404641939ec7d93266fcca" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34831 } } peers { permanent_uuid: "9e45adfc7114482c9bfd1cc0b6b75a96" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 40721 } }
I20260806 14:58:39.583709 21419 raft_consensus.cc:374] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:39.583637 21417 raft_consensus.cc:348] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 9e45adfc7114482c9bfd1cc0b6b75a96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd4e5e034bb647b2adf0bbf9f4850843" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 46791 } } peers { permanent_uuid: "7887589cb7404641939ec7d93266fcca" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34831 } } peers { permanent_uuid: "9e45adfc7114482c9bfd1cc0b6b75a96" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 40721 } }
I20260806 14:58:39.583752 21419 raft_consensus.cc:729] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7887589cb7404641939ec7d93266fcca, State: Initialized, Role: FOLLOWER
I20260806 14:58:39.583798 21417 raft_consensus.cc:374] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 9e45adfc7114482c9bfd1cc0b6b75a96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:39.583843 21417 raft_consensus.cc:729] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 9e45adfc7114482c9bfd1cc0b6b75a96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e45adfc7114482c9bfd1cc0b6b75a96, State: Initialized, Role: FOLLOWER
I20260806 14:58:39.583887 21419 consensus_queue.cc:260] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd4e5e034bb647b2adf0bbf9f4850843" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 46791 } } peers { permanent_uuid: "7887589cb7404641939ec7d93266fcca" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34831 } } peers { permanent_uuid: "9e45adfc7114482c9bfd1cc0b6b75a96" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 40721 } }
I20260806 14:58:39.583954 21417 consensus_queue.cc:260] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 9e45adfc7114482c9bfd1cc0b6b75a96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd4e5e034bb647b2adf0bbf9f4850843" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 46791 } } peers { permanent_uuid: "7887589cb7404641939ec7d93266fcca" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34831 } } peers { permanent_uuid: "9e45adfc7114482c9bfd1cc0b6b75a96" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 40721 } }
I20260806 14:58:39.584190 21419 ts_tablet_manager.cc:1428] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 14:58:39.584539 21360 heartbeater.cc:503] Master 127.11.8.126:34611 was elected leader, sending a full tablet report...
I20260806 14:58:39.584302 20974 heartbeater.cc:503] Master 127.11.8.126:34611 was elected leader, sending a full tablet report...
I20260806 14:58:39.584230 21417 ts_tablet_manager.cc:1428] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 9e45adfc7114482c9bfd1cc0b6b75a96: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260806 14:58:39.586786 21420 tablet_bootstrap.cc:492] T 78cee5e85c3f41fbb91ddbf28e761dd6 P cd4e5e034bb647b2adf0bbf9f4850843: No bootstrap required, opened a new log
I20260806 14:58:39.586844 21420 ts_tablet_manager.cc:1397] T 78cee5e85c3f41fbb91ddbf28e761dd6 P cd4e5e034bb647b2adf0bbf9f4850843: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260806 14:58:39.588194 21420 raft_consensus.cc:348] T 78cee5e85c3f41fbb91ddbf28e761dd6 P cd4e5e034bb647b2adf0bbf9f4850843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd4e5e034bb647b2adf0bbf9f4850843" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 46791 } } peers { permanent_uuid: "7887589cb7404641939ec7d93266fcca" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34831 } } peers { permanent_uuid: "9e45adfc7114482c9bfd1cc0b6b75a96" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 40721 } }
I20260806 14:58:39.588320 21420 raft_consensus.cc:374] T 78cee5e85c3f41fbb91ddbf28e761dd6 P cd4e5e034bb647b2adf0bbf9f4850843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:39.588341 21420 raft_consensus.cc:729] T 78cee5e85c3f41fbb91ddbf28e761dd6 P cd4e5e034bb647b2adf0bbf9f4850843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd4e5e034bb647b2adf0bbf9f4850843, State: Initialized, Role: FOLLOWER
I20260806 14:58:39.588485 21420 consensus_queue.cc:260] T 78cee5e85c3f41fbb91ddbf28e761dd6 P cd4e5e034bb647b2adf0bbf9f4850843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd4e5e034bb647b2adf0bbf9f4850843" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 46791 } } peers { permanent_uuid: "7887589cb7404641939ec7d93266fcca" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34831 } } peers { permanent_uuid: "9e45adfc7114482c9bfd1cc0b6b75a96" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 40721 } }
I20260806 14:58:39.588752 21420 ts_tablet_manager.cc:1428] T 78cee5e85c3f41fbb91ddbf28e761dd6 P cd4e5e034bb647b2adf0bbf9f4850843: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 14:58:39.588954 21165 heartbeater.cc:503] Master 127.11.8.126:34611 was elected leader, sending a full tablet report...
W20260806 14:58:39.628659 21167 tablet.cc:2369] T 78cee5e85c3f41fbb91ddbf28e761dd6 P cd4e5e034bb647b2adf0bbf9f4850843: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 14:58:39.715746 20978 tablet.cc:2369] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 9e45adfc7114482c9bfd1cc0b6b75a96: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 14:58:39.785208 21366 tablet.cc:2369] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 14:58:39.817756 21424 raft_consensus.cc:482] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 14:58:39.817919 21424 raft_consensus.cc:504] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd4e5e034bb647b2adf0bbf9f4850843" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 46791 } } peers { permanent_uuid: "7887589cb7404641939ec7d93266fcca" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34831 } } peers { permanent_uuid: "9e45adfc7114482c9bfd1cc0b6b75a96" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 40721 } }
I20260806 14:58:39.818806 21424 leader_election.cc:288] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd4e5e034bb647b2adf0bbf9f4850843 (127.11.8.66:46791), 9e45adfc7114482c9bfd1cc0b6b75a96 (127.11.8.65:40721)
I20260806 14:58:39.824290 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.818764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:40721 (local address 127.11.8.67:41577)
0806 14:58:39.818798 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:39.818807 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:39.818834 (+ 27us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:39.819138 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:39.819139 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:39.819142 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:39.819233 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.819235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.820277 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.820279 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.821601 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.821605 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.824009 (+ 2404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.824010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.824097 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.824101 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 14:58:39.824171 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 14:58:39.824216 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260806 14:58:39.824339 21416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.818877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:41577 (local address 127.11.8.65:40721)
0806 14:58:39.818957 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:39.818959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:39.818969 (+ 10us) server_negotiation.cc:407] Connection header received
0806 14:58:39.818998 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:39.818999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:39.819028 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:39.819096 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:39.819333 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.820170 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.823269 (+ 3099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.823918 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.824002 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.824013 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:39.824179 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 14:58:39.824234 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 14:58:39.824769 20905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cee5e85c3f41fbb91ddbf28e761dd6" candidate_uuid: "7887589cb7404641939ec7d93266fcca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e45adfc7114482c9bfd1cc0b6b75a96" is_pre_election: true
I20260806 14:58:39.824944 20905 raft_consensus.cc:2454] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 9e45adfc7114482c9bfd1cc0b6b75a96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7887589cb7404641939ec7d93266fcca in term 0.
I20260806 14:58:39.825227 21193 leader_election.cc:302] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7887589cb7404641939ec7d93266fcca, 9e45adfc7114482c9bfd1cc0b6b75a96; no voters:
I20260806 14:58:39.825384 21424 raft_consensus.cc:2781] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 14:58:39.825444 21424 raft_consensus.cc:482] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 14:58:39.825475 21424 raft_consensus.cc:3037] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:39.826277 21424 raft_consensus.cc:504] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd4e5e034bb647b2adf0bbf9f4850843" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 46791 } } peers { permanent_uuid: "7887589cb7404641939ec7d93266fcca" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34831 } } peers { permanent_uuid: "9e45adfc7114482c9bfd1cc0b6b75a96" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 40721 } }
I20260806 14:58:39.826483 21424 leader_election.cc:288] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca [CANDIDATE]: Term 1 election: Requested vote from peers cd4e5e034bb647b2adf0bbf9f4850843 (127.11.8.66:46791), 9e45adfc7114482c9bfd1cc0b6b75a96 (127.11.8.65:40721)
I20260806 14:58:39.826494 21411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.823311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:38061 (local address 127.11.8.66:46791)
0806 14:58:39.823355 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:39.823358 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:39.823365 (+ 7us) server_negotiation.cc:407] Connection header received
0806 14:58:39.823392 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:39.823393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:39.823404 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:39.823484 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:39.823656 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.824294 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.825614 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.826103 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.826199 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.826210 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:39.826370 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 14:58:39.826432 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 14:58:39.826510 21367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.818566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:46791 (local address 127.11.8.67:38061)
0806 14:58:39.818625 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:39.818642 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:39.818711 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:39.823501 (+ 4790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:39.823503 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:39.823505 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:39.823560 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.823561 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.824390 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.824391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.825506 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:39.825509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.826199 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.826200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:39.826272 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.826275 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 14:58:39.826360 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 14:58:39.826398 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 14:58:39.826656 20905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cee5e85c3f41fbb91ddbf28e761dd6" candidate_uuid: "7887589cb7404641939ec7d93266fcca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e45adfc7114482c9bfd1cc0b6b75a96"
I20260806 14:58:39.826737 20905 raft_consensus.cc:3037] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 9e45adfc7114482c9bfd1cc0b6b75a96 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:39.826825 21099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cee5e85c3f41fbb91ddbf28e761dd6" candidate_uuid: "7887589cb7404641939ec7d93266fcca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4e5e034bb647b2adf0bbf9f4850843" is_pre_election: true
I20260806 14:58:39.826834 21098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78cee5e85c3f41fbb91ddbf28e761dd6" candidate_uuid: "7887589cb7404641939ec7d93266fcca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4e5e034bb647b2adf0bbf9f4850843"
I20260806 14:58:39.826992 21099 raft_consensus.cc:2454] T 78cee5e85c3f41fbb91ddbf28e761dd6 P cd4e5e034bb647b2adf0bbf9f4850843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7887589cb7404641939ec7d93266fcca in term 0.
I20260806 14:58:39.827672 20905 raft_consensus.cc:2454] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 9e45adfc7114482c9bfd1cc0b6b75a96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7887589cb7404641939ec7d93266fcca in term 1.
I20260806 14:58:39.827857 21193 leader_election.cc:302] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7887589cb7404641939ec7d93266fcca, 9e45adfc7114482c9bfd1cc0b6b75a96; no voters: cd4e5e034bb647b2adf0bbf9f4850843
I20260806 14:58:39.827975 21424 raft_consensus.cc:2781] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:39.828111 21424 raft_consensus.cc:686] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca [term 1 LEADER]: Becoming Leader. State: Replica: 7887589cb7404641939ec7d93266fcca, State: Running, Role: LEADER
I20260806 14:58:39.828190 21424 consensus_queue.cc:237] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd4e5e034bb647b2adf0bbf9f4850843" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 46791 } } peers { permanent_uuid: "7887589cb7404641939ec7d93266fcca" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34831 } } peers { permanent_uuid: "9e45adfc7114482c9bfd1cc0b6b75a96" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 40721 } }
I20260806 14:58:39.829254 19637 catalog_manager.cc:5495] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca reported cstate change: term changed from 0 to 1, leader changed from <none> to 7887589cb7404641939ec7d93266fcca (127.11.8.67). New cstate: current_term: 1 leader_uuid: "7887589cb7404641939ec7d93266fcca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd4e5e034bb647b2adf0bbf9f4850843" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 46791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7887589cb7404641939ec7d93266fcca" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e45adfc7114482c9bfd1cc0b6b75a96" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 40721 } health_report { overall_health: UNKNOWN } } }
I20260806 14:58:39.853929 21415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.851315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56832 (local address 127.11.8.67:34831)
0806 14:58:39.851380 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:39.851382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:39.851390 (+ 8us) server_negotiation.cc:407] Connection header received
0806 14:58:39.851461 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:39.851461 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:39.851482 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:39.851542 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 14:58:39.851710 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.852214 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.852988 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.853270 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.853342 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.853561 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 14:58:39.853674 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 14:58:39.853753 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:39.853799 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 14:58:39.853821 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 14:58:39.856196 21099 raft_consensus.cc:3037] T 78cee5e85c3f41fbb91ddbf28e761dd6 P cd4e5e034bb647b2adf0bbf9f4850843 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:39.856277 20905 raft_consensus.cc:1261] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 9e45adfc7114482c9bfd1cc0b6b75a96 [term 1 FOLLOWER]: Refusing update from remote peer 7887589cb7404641939ec7d93266fcca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:39.856671 21424 consensus_queue.cc:1035] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e45adfc7114482c9bfd1cc0b6b75a96" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:39.858099 21099 raft_consensus.cc:1261] T 78cee5e85c3f41fbb91ddbf28e761dd6 P cd4e5e034bb647b2adf0bbf9f4850843 [term 1 FOLLOWER]: Refusing update from remote peer 7887589cb7404641939ec7d93266fcca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:39.858592 21424 consensus_queue.cc:1035] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd4e5e034bb647b2adf0bbf9f4850843" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 46791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:39.859385 21435 mvcc.cc:205] Tried to move back new op lower bound from 7315571998127591424 to 7315571998017527808. Current Snapshot: MvccSnapshot[applied={T|T < 7315571998127591424}]
I20260806 14:58:39.862234 21443 mvcc.cc:205] Tried to move back new op lower bound from 7315571998127591424 to 7315571998017527808. Current Snapshot: MvccSnapshot[applied={T|T < 7315571998127591424}]
I20260806 14:58:39.863327 19637 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:58:39.863416 19637 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:58:39.864622 19637 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d5ce68764a334405bdb27775b2321b6a: Sending DeleteTablet for 3 replicas of tablet 78cee5e85c3f41fbb91ddbf28e761dd6
I20260806 14:58:39.864967 21265 tablet_service.cc:1514] Processing DeleteTablet for tablet 78cee5e85c3f41fbb91ddbf28e761dd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:39 UTC) from {username='oryx', principal='oryx/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:56828
I20260806 14:58:39.865003 21070 tablet_service.cc:1514] Processing DeleteTablet for tablet 78cee5e85c3f41fbb91ddbf28e761dd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:39 UTC) from {username='oryx', principal='oryx/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:53160
I20260806 14:58:39.865218 20878 tablet_service.cc:1514] Processing DeleteTablet for tablet 78cee5e85c3f41fbb91ddbf28e761dd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:39 UTC) from {username='oryx', principal='oryx/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:45418
I20260806 14:58:39.865414 21449 tablet_replica.cc:330] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 9e45adfc7114482c9bfd1cc0b6b75a96: stopping tablet replica
I20260806 14:58:39.865417 21448 tablet_replica.cc:330] T 78cee5e85c3f41fbb91ddbf28e761dd6 P cd4e5e034bb647b2adf0bbf9f4850843: stopping tablet replica
I20260806 14:58:39.865507 21448 raft_consensus.cc:2229] T 78cee5e85c3f41fbb91ddbf28e761dd6 P cd4e5e034bb647b2adf0bbf9f4850843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:58:39.865507 21449 raft_consensus.cc:2229] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 9e45adfc7114482c9bfd1cc0b6b75a96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:58:39.865607 21449 raft_consensus.cc:2258] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 9e45adfc7114482c9bfd1cc0b6b75a96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:39.865607 21448 raft_consensus.cc:2258] T 78cee5e85c3f41fbb91ddbf28e761dd6 P cd4e5e034bb647b2adf0bbf9f4850843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:39.865839 21447 tablet_replica.cc:330] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca: stopping tablet replica
I20260806 14:58:39.865935 21447 raft_consensus.cc:2229] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca [term 1 LEADER]: Raft consensus shutting down.
I20260806 14:58:39.866010 21447 raft_consensus.cc:2258] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:39.866036 21449 ts_tablet_manager.cc:1905] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 9e45adfc7114482c9bfd1cc0b6b75a96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:39.866391 21447 ts_tablet_manager.cc:1905] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:39.867195 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 19593
I20260806 14:58:39.876048 21449 ts_tablet_manager.cc:1918] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 9e45adfc7114482c9bfd1cc0b6b75a96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 14:58:39.876149 21449 log.cc:1196] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 9e45adfc7114482c9bfd1cc0b6b75a96: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-0/wal/wals/78cee5e85c3f41fbb91ddbf28e761dd6
I20260806 14:58:39.876446 21449 ts_tablet_manager.cc:1939] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 9e45adfc7114482c9bfd1cc0b6b75a96: Deleting consensus metadata
I20260806 14:58:39.876883 21447 ts_tablet_manager.cc:1918] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 14:58:39.876964 21447 log.cc:1196] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/wal/wals/78cee5e85c3f41fbb91ddbf28e761dd6
I20260806 14:58:39.877233 21447 ts_tablet_manager.cc:1939] T 78cee5e85c3f41fbb91ddbf28e761dd6 P 7887589cb7404641939ec7d93266fcca: Deleting consensus metadata
I20260806 14:58:39.879191 21448 ts_tablet_manager.cc:1905] T 78cee5e85c3f41fbb91ddbf28e761dd6 P cd4e5e034bb647b2adf0bbf9f4850843: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 14:58:39.880823 20814 connection.cc:535] server connection from 127.0.0.1:45418 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 14:58:39.880831 21194 connection.cc:535] server connection from 127.0.0.1:56828 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 14:58:39.880914 21448 ts_tablet_manager.cc:1918] T 78cee5e85c3f41fbb91ddbf28e761dd6 P cd4e5e034bb647b2adf0bbf9f4850843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 14:58:39.881147 21448 log.cc:1196] T 78cee5e85c3f41fbb91ddbf28e761dd6 P cd4e5e034bb647b2adf0bbf9f4850843: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/wal/wals/78cee5e85c3f41fbb91ddbf28e761dd6
I20260806 14:58:39.881472 21448 ts_tablet_manager.cc:1939] T 78cee5e85c3f41fbb91ddbf28e761dd6 P cd4e5e034bb647b2adf0bbf9f4850843: Deleting consensus metadata
W20260806 14:58:39.881863 21006 connection.cc:441] server connection from 127.0.0.1:53160 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53160 (request call id 1) could send its response
I20260806 14:58:39.890590 20530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.883218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42632 (local address 127.11.8.125:42909)
0806 14:58:39.883266 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:39.883268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:39.883396 (+ 128us) server_negotiation.cc:407] Connection header received
0806 14:58:39.883488 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:39.883489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:39.883500 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:39.883533 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:39.884196 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.884688 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.885511 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.885717 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.885793 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.887420 (+ 1627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:39.887421 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:39.887422 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:39.887432 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:39.889064 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.889525 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.889526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.889527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.889592 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.889844 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.889845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.889846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.889983 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:39.890093 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:39.890256 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 14:58:39.890334 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 14:58:39.891539 21376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:39.883093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34178 (local address 127.11.8.124:34581)
0806 14:58:39.883142 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:39.883145 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:39.883337 (+ 192us) server_negotiation.cc:407] Connection header received
0806 14:58:39.883468 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:39.883469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:39.883482 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:39.883539 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:39.884385 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.885253 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.886494 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:39.886803 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:39.886887 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:39.888642 (+ 1755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:39.888643 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:39.888643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:39.888651 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:39.890368 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.890789 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.890790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.890791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.890852 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:39.891077 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:39.891078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:39.891079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:39.891191 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:39.891266 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:39.891421 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 14:58:39.891480 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 14:58:40.862617 21459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:40.862277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.11.8.67:39759)
0806 14:58:40.862459 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:40.862546 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":71,"threads_started":1}
W20260806 14:58:40.862943 21360 heartbeater.cc:644] Failed to heartbeat to 127.11.8.126:34611 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
I20260806 14:58:40.863125 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:40.862861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.11.8.65:34945)
0806 14:58:40.862997 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:40.863073 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":60,"threads_started":1}
W20260806 14:58:40.863368 20974 heartbeater.cc:644] Failed to heartbeat to 127.11.8.126:34611 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
I20260806 14:58:40.863677 21461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:40.863404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.11.8.66:60937)
0806 14:58:40.863555 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:40.863626 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":81,"threads_started":1}
W20260806 14:58:40.863844 21165 heartbeater.cc:644] Failed to heartbeat to 127.11.8.126:34611 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
I20260806 14:58:41.365386 21463 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d5ce68764a334405bdb27775b2321b6a)
I20260806 14:58:41.365577 21463 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: false } }
I20260806 14:58:41.365684 21464 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d5ce68764a334405bdb27775b2321b6a)
I20260806 14:58:41.365911 21463 leader_election.cc:288] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d5ce68764a334405bdb27775b2321b6a (127.11.8.126:34611), ef653e0dfa3841999c9d8f70b12f03e6 (127.11.8.125:42909)
I20260806 14:58:41.365847 21464 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: false } }
I20260806 14:58:41.366171 21464 leader_election.cc:288] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d5ce68764a334405bdb27775b2321b6a (127.11.8.126:34611), ed19be708aac49f982e3e226c5180acd (127.11.8.124:34581)
I20260806 14:58:41.366460 21466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:41.366147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.0.0.1:54712)
0806 14:58:41.366307 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:41.366367 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":75,"threads_started":1}
I20260806 14:58:41.366597 21469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:41.366298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.0.0.1:54722)
0806 14:58:41.366504 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:41.366549 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":66,"threads_started":1}
W20260806 14:58:41.366612 20700 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
W20260806 14:58:41.366706 20443 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
I20260806 14:58:41.367074 21466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:41.366969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.0.0.1:54732)
0806 14:58:41.367007 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:41.367038 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260806 14:58:41.367168 20700 leader_election.cc:334] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d5ce68764a334405bdb27775b2321b6a (127.11.8.126:34611): Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
I20260806 14:58:41.367324 21470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:41.367024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.0.0.1:54738)
0806 14:58:41.367103 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:41.367289 (+ 186us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260806 14:58:41.367430 20443 leader_election.cc:334] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d5ce68764a334405bdb27775b2321b6a (127.11.8.126:34611): Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
I20260806 14:58:41.369863 21469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:41.366314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:34581 (local address 127.0.0.1:34506)
0806 14:58:41.366877 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:41.366888 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:41.367059 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:41.367316 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:41.367316 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:41.367319 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:41.367354 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:41.367355 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:41.368222 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:41.368223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:41.368971 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:41.368973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:41.369517 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:41.369518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:41.369585 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:41.369590 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 14:58:41.369731 (+ 141us) client_negotiation.cc:240] Negotiation successful
0806 14:58:41.369794 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"mutex_wait_us":3,"thread_start_us":48,"threads_started":1}
I20260806 14:58:41.369884 21474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:41.366540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34506 (local address 127.11.8.124:34581)
0806 14:58:41.366798 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:41.366800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:41.367048 (+ 248us) server_negotiation.cc:407] Connection header received
0806 14:58:41.367147 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:41.367148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:41.367159 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:41.367254 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:41.367424 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:41.368128 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:41.369068 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:41.369435 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:41.369518 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:41.369530 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:41.369758 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 14:58:41.369826 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":55,"threads_started":1}
I20260806 14:58:41.370052 21471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:41.366355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:42909 (local address 127.0.0.1:55864)
0806 14:58:41.366623 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:41.366636 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:41.366702 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:41.367032 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:41.367034 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:41.367037 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:41.367107 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:41.367110 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:41.368197 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:41.368198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:41.369031 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:41.369033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:41.369716 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:41.369717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:41.369794 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:41.369799 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 14:58:41.369884 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 14:58:41.369936 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":1,"thread_start_us":94,"threads_started":1}
I20260806 14:58:41.370119 20737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ed19be708aac49f982e3e226c5180acd" is_pre_election: true
I20260806 14:58:41.370234 20737 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef653e0dfa3841999c9d8f70b12f03e6 in term 1.
I20260806 14:58:41.370253 21473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:41.366504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55864 (local address 127.11.8.125:42909)
0806 14:58:41.366869 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:41.366872 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:41.366882 (+ 10us) server_negotiation.cc:407] Connection header received
0806 14:58:41.366913 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:41.366914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:41.366926 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:41.367230 (+ 304us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:41.367303 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:41.368102 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:41.369189 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:41.369618 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:41.369715 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:41.369727 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:41.369894 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 14:58:41.370080 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":56,"threads_started":1}
I20260806 14:58:41.370399 20442 leader_election.cc:302] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ed19be708aac49f982e3e226c5180acd, ef653e0dfa3841999c9d8f70b12f03e6; no voters: d5ce68764a334405bdb27775b2321b6a
I20260806 14:58:41.370491 20475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ed19be708aac49f982e3e226c5180acd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" is_pre_election: true
I20260806 14:58:41.370548 21464 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 14:58:41.370599 20475 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed19be708aac49f982e3e226c5180acd in term 1.
I20260806 14:58:41.370680 21464 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d5ce68764a334405bdb27775b2321b6a)
I20260806 14:58:41.370721 21464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 1 FOLLOWER]: Advancing to term 2
I20260806 14:58:41.370783 20700 leader_election.cc:302] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ed19be708aac49f982e3e226c5180acd, ef653e0dfa3841999c9d8f70b12f03e6; no voters: d5ce68764a334405bdb27775b2321b6a
I20260806 14:58:41.370893 21463 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 14:58:41.370932 21463 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [term 1 FOLLOWER]: Starting leader election (detected failure of leader d5ce68764a334405bdb27775b2321b6a)
I20260806 14:58:41.370959 21463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [term 1 FOLLOWER]: Advancing to term 2
I20260806 14:58:41.371551 21464 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: false } }
I20260806 14:58:41.371635 21463 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: false } }
I20260806 14:58:41.371712 21464 leader_election.cc:288] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [CANDIDATE]: Term 2 election: Requested vote from peers d5ce68764a334405bdb27775b2321b6a (127.11.8.126:34611), ed19be708aac49f982e3e226c5180acd (127.11.8.124:34581)
I20260806 14:58:41.371779 21463 leader_election.cc:288] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [CANDIDATE]: Term 2 election: Requested vote from peers d5ce68764a334405bdb27775b2321b6a (127.11.8.126:34611), ef653e0dfa3841999c9d8f70b12f03e6 (127.11.8.125:42909)
I20260806 14:58:41.371891 21469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:41.371793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.0.0.1:54748)
0806 14:58:41.371830 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:41.371860 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260806 14:58:41.371990 21471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:41.371860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.0.0.1:54756)
0806 14:58:41.371938 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:41.371965 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260806 14:58:41.371999 20475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ed19be708aac49f982e3e226c5180acd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ef653e0dfa3841999c9d8f70b12f03e6"
I20260806 14:58:41.371999 20737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ed19be708aac49f982e3e226c5180acd"
I20260806 14:58:41.372093 20737 raft_consensus.cc:2379] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ef653e0dfa3841999c9d8f70b12f03e6 in current term 2: Already voted for candidate ed19be708aac49f982e3e226c5180acd in this term.
I20260806 14:58:41.372094 20475 raft_consensus.cc:2379] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ed19be708aac49f982e3e226c5180acd in current term 2: Already voted for candidate ef653e0dfa3841999c9d8f70b12f03e6 in this term.
I20260806 14:58:41.372336 21469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:41.372236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.0.0.1:54762)
0806 14:58:41.372276 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:41.372308 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260806 14:58:41.372466 20443 leader_election.cc:334] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d5ce68764a334405bdb27775b2321b6a (127.11.8.126:34611): Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
I20260806 14:58:41.372547 20443 leader_election.cc:302] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ef653e0dfa3841999c9d8f70b12f03e6; no voters: d5ce68764a334405bdb27775b2321b6a, ed19be708aac49f982e3e226c5180acd
I20260806 14:58:41.372617 21471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:41.372501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.0.0.1:54764)
0806 14:58:41.372566 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:41.372590 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 14:58:41.372694 21464 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260806 14:58:41.372709 20700 leader_election.cc:334] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d5ce68764a334405bdb27775b2321b6a (127.11.8.126:34611): Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
I20260806 14:58:41.372757 20700 leader_election.cc:302] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed19be708aac49f982e3e226c5180acd; no voters: d5ce68764a334405bdb27775b2321b6a, ef653e0dfa3841999c9d8f70b12f03e6
I20260806 14:58:41.372884 21463 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 14:58:41.863618 21478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:41.863308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.11.8.67:34871)
0806 14:58:41.863486 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:41.863558 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":96,"threads_started":1}
I20260806 14:58:41.864034 21481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:41.863685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.11.8.65:44683)
0806 14:58:41.863920 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:41.863970 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":51,"threads_started":1}
I20260806 14:58:41.864359 21482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:41.864110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.11.8.66:43457)
0806 14:58:41.864279 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:41.864311 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":69,"threads_started":1}
I20260806 14:58:42.864283 21483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:42.864051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.11.8.67:42981)
0806 14:58:42.864175 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:42.864221 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":77,"threads_started":1}
I20260806 14:58:42.864741 21484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:42.864528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.11.8.65:34481)
0806 14:58:42.864651 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:42.864695 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
I20260806 14:58:42.864938 21485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:42.864724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.11.8.66:34155)
0806 14:58:42.864856 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:42.864890 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260806 14:58:43.059378 21486 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 14:58:43.059470 21486 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: false } }
I20260806 14:58:43.059624 21486 leader_election.cc:288] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d5ce68764a334405bdb27775b2321b6a (127.11.8.126:34611), ed19be708aac49f982e3e226c5180acd (127.11.8.124:34581)
I20260806 14:58:43.059860 20737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ed19be708aac49f982e3e226c5180acd" is_pre_election: true
I20260806 14:58:43.059948 20737 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef653e0dfa3841999c9d8f70b12f03e6 in term 2.
I20260806 14:58:43.059973 21487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.059744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.0.0.1:55008)
0806 14:58:43.059868 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:43.059911 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
I20260806 14:58:43.060127 20442 leader_election.cc:302] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ed19be708aac49f982e3e226c5180acd, ef653e0dfa3841999c9d8f70b12f03e6; no voters:
I20260806 14:58:43.060276 21486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 14:58:43.060331 21486 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 14:58:43.060353 21486 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 2 FOLLOWER]: Advancing to term 3
I20260806 14:58:43.060531 21487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.060432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.0.0.1:55012)
0806 14:58:43.060471 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:43.060500 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260806 14:58:43.060607 20443 leader_election.cc:334] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d5ce68764a334405bdb27775b2321b6a (127.11.8.126:34611): Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
I20260806 14:58:43.061255 21486 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: false } }
I20260806 14:58:43.061403 21486 leader_election.cc:288] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [CANDIDATE]: Term 3 election: Requested vote from peers d5ce68764a334405bdb27775b2321b6a (127.11.8.126:34611), ed19be708aac49f982e3e226c5180acd (127.11.8.124:34581)
I20260806 14:58:43.061564 21487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.061469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.0.0.1:55024)
0806 14:58:43.061506 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:43.061541 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260806 14:58:43.061553 20737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ed19be708aac49f982e3e226c5180acd"
I20260806 14:58:43.061623 20737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [term 2 FOLLOWER]: Advancing to term 3
I20260806 14:58:43.061909 21487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.061794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.0.0.1:55030)
0806 14:58:43.061857 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:43.061888 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 14:58:43.062032 20443 leader_election.cc:334] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d5ce68764a334405bdb27775b2321b6a (127.11.8.126:34611): Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
I20260806 14:58:43.062477 20737 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef653e0dfa3841999c9d8f70b12f03e6 in term 3.
I20260806 14:58:43.062654 20442 leader_election.cc:302] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ed19be708aac49f982e3e226c5180acd, ef653e0dfa3841999c9d8f70b12f03e6; no voters: d5ce68764a334405bdb27775b2321b6a
I20260806 14:58:43.062772 21486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 14:58:43.062927 21486 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [term 3 LEADER]: Becoming Leader. State: Replica: ef653e0dfa3841999c9d8f70b12f03e6, State: Running, Role: LEADER
I20260806 14:58:43.063009 21486 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: false } }
I20260806 14:58:43.063377 21491 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef653e0dfa3841999c9d8f70b12f03e6. Latest consensus state: current_term: 3 leader_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: false } } }
I20260806 14:58:43.063479 21491 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:43.063678 21493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 14:58:43.064174 21493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 14:58:43.064280 21493 catalog_manager.cc:1259] Loaded cluster ID: c461a7c22dcb49a6a9a5e229a9598c26
I20260806 14:58:43.064317 21493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 14:58:43.064500 21493 catalog_manager.cc:1504] Loading token signing keys...
I20260806 14:58:43.064594 21493 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6: Loaded TSK: 0
I20260806 14:58:43.064749 21493 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 14:58:43.142889 20462 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42632:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 14:58:43.144778 20462 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 14:58:43.145838 21487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.145714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.0.0.1:55050)
0806 14:58:43.145773 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:43.145811 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 14:58:43.145851 20737 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [term 3 FOLLOWER]: Refusing update from remote peer ef653e0dfa3841999c9d8f70b12f03e6: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260806 14:58:43.146138 21491 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260806 14:58:43.146243 21487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.146139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.0.0.1:55062)
0806 14:58:43.146191 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:43.146222 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 14:58:43.146350 20443 consensus_peers.cc:487] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 -> Peer d5ce68764a334405bdb27775b2321b6a (127.11.8.126:34611): Couldn't send request to peer d5ce68764a334405bdb27775b2321b6a. Status: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 14:58:43.146834 21495 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef653e0dfa3841999c9d8f70b12f03e6. Latest consensus state: current_term: 3 leader_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: false } } }
I20260806 14:58:43.146955 21495 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 14:58:43.147466 21497 mvcc.cc:205] Tried to move back new op lower bound from 7315572011603398656 to 7315572011266768896. Current Snapshot: MvccSnapshot[applied={T|T < 7315572011603398656}]
I20260806 14:58:43.147440 21491 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: false } } }
I20260806 14:58:43.147547 21491 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6 [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:43.147744 21495 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "d5ce68764a334405bdb27775b2321b6a" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34611 } } peers { permanent_uuid: "ef653e0dfa3841999c9d8f70b12f03e6" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 42909 } attrs { promote: false } } peers { permanent_uuid: "ed19be708aac49f982e3e226c5180acd" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 34581 } attrs { promote: false } } }
I20260806 14:58:43.147846 21495 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed19be708aac49f982e3e226c5180acd [sys.catalog]: This master's current role is: FOLLOWER
I20260806 14:58:43.152598 21506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.149436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49586 (local address 127.11.8.66:46791)
0806 14:58:43.149551 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:43.149553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:43.149560 (+ 7us) server_negotiation.cc:407] Connection header received
0806 14:58:43.149587 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:43.149588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:43.149600 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:43.149675 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:43.149867 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:43.150536 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:43.151722 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:43.152289 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:43.152359 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:43.152372 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:43.152489 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 14:58:43.152534 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":60,"threads_started":1}
I20260806 14:58:43.152593 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.149332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:46791 (local address 127.0.0.1:49586)
0806 14:58:43.149438 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:43.149447 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:43.149488 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:43.149728 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:43.149729 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:43.149732 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:43.149785 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:43.149787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:43.150614 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:43.150615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:43.151621 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:43.151624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:43.152357 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:43.152358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:43.152417 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:43.152421 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 14:58:43.152482 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 14:58:43.152523 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":60,"threads_started":1}
I20260806 14:58:43.153014 21487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.149229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:34831 (local address 127.0.0.1:58030)
0806 14:58:43.149259 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:43.149269 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:43.149324 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:43.150067 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:43.150068 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:43.150070 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:43.150105 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:43.150107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:43.151060 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:43.151061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:43.152007 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:43.152009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:43.152799 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:43.152800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:43.152859 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:43.152862 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 14:58:43.152909 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 14:58:43.152950 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15,"mutex_wait_us":45}
I20260806 14:58:43.153053 21508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.149636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58030 (local address 127.11.8.67:34831)
0806 14:58:43.149894 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:43.149896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:43.149904 (+ 8us) server_negotiation.cc:407] Connection header received
0806 14:58:43.149931 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:43.149932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:43.149943 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:43.150048 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:43.150187 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:43.150953 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:43.152095 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:43.152713 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:43.152788 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:43.152800 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:43.152930 (+ 130us) server_negotiation.cc:299] Negotiation successful
0806 14:58:43.152981 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":62,"threads_started":1}
I20260806 14:58:43.153209 21507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.149489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:40721 (local address 127.0.0.1:34324)
0806 14:58:43.149602 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:43.149615 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:43.149671 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:43.150091 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:43.150092 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:43.150094 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:43.150138 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:43.150139 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:43.151062 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:43.151063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:43.152208 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:43.152210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:43.152990 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:43.152990 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:43.153050 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:43.153054 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 14:58:43.153085 (+ 31us) client_negotiation.cc:240] Negotiation successful
0806 14:58:43.153129 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":28,"thread_start_us":87,"threads_started":1}
I20260806 14:58:43.153280 21070 tablet_service.cc:1467] Processing CreateTablet for tablet 4fbcc7daf7da4af0809d9e1a54c3cc26 (DEFAULT_TABLE table=test-table [id=c02118425c8d4812b3b43eb1c0e42293]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:43.153424 21070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fbcc7daf7da4af0809d9e1a54c3cc26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:43.153623 21509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.149756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34324 (local address 127.11.8.65:40721)
0806 14:58:43.149892 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:43.149895 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:43.149904 (+ 9us) server_negotiation.cc:407] Connection header received
0806 14:58:43.149931 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:43.149932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:43.149943 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:43.150210 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:43.150252 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:43.150978 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:43.152290 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:43.152875 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:43.153455 (+ 580us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:43.153472 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:43.153494 (+ 22us) server_negotiation.cc:299] Negotiation successful
0806 14:58:43.153545 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":63,"threads_started":1}
I20260806 14:58:43.153844 20878 tablet_service.cc:1467] Processing CreateTablet for tablet 4fbcc7daf7da4af0809d9e1a54c3cc26 (DEFAULT_TABLE table=test-table [id=c02118425c8d4812b3b43eb1c0e42293]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:43.153957 20878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fbcc7daf7da4af0809d9e1a54c3cc26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:43.154778 21510 tablet_bootstrap.cc:492] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843: Bootstrap starting.
I20260806 14:58:43.155038 21265 tablet_service.cc:1467] Processing CreateTablet for tablet 4fbcc7daf7da4af0809d9e1a54c3cc26 (DEFAULT_TABLE table=test-table [id=c02118425c8d4812b3b43eb1c0e42293]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:43.155144 21265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fbcc7daf7da4af0809d9e1a54c3cc26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:43.155174 21510 tablet_bootstrap.cc:654] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:43.155212 21512 tablet_bootstrap.cc:492] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 9e45adfc7114482c9bfd1cc0b6b75a96: Bootstrap starting.
I20260806 14:58:43.155570 21512 tablet_bootstrap.cc:654] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 9e45adfc7114482c9bfd1cc0b6b75a96: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:43.155972 21510 tablet_bootstrap.cc:492] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843: No bootstrap required, opened a new log
I20260806 14:58:43.156038 21510 ts_tablet_manager.cc:1397] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:43.156172 21510 raft_consensus.cc:348] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7887589cb7404641939ec7d93266fcca" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34831 } } peers { permanent_uuid: "cd4e5e034bb647b2adf0bbf9f4850843" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 46791 } } peers { permanent_uuid: "9e45adfc7114482c9bfd1cc0b6b75a96" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 40721 } }
I20260806 14:58:43.156232 21510 raft_consensus.cc:374] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:43.156262 21510 raft_consensus.cc:729] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd4e5e034bb647b2adf0bbf9f4850843, State: Initialized, Role: FOLLOWER
I20260806 14:58:43.156312 21510 consensus_queue.cc:260] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7887589cb7404641939ec7d93266fcca" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34831 } } peers { permanent_uuid: "cd4e5e034bb647b2adf0bbf9f4850843" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 46791 } } peers { permanent_uuid: "9e45adfc7114482c9bfd1cc0b6b75a96" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 40721 } }
I20260806 14:58:43.156922 21485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.156772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.11.8.66:45935)
0806 14:58:43.156867 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:43.156897 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 14:58:43.157197 21510 ts_tablet_manager.cc:1428] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:43.157377 21517 tablet_bootstrap.cc:492] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 7887589cb7404641939ec7d93266fcca: Bootstrap starting.
I20260806 14:58:43.157732 21166 heartbeater.cc:503] Master 127.11.8.125:42909 was elected leader, sending a full tablet report...
I20260806 14:58:43.157868 21485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.157658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.11.8.66:46649)
0806 14:58:43.157693 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:43.157722 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260806 14:58:43.157930 21517 tablet_bootstrap.cc:654] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 7887589cb7404641939ec7d93266fcca: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:43.158591 21517 tablet_bootstrap.cc:492] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 7887589cb7404641939ec7d93266fcca: No bootstrap required, opened a new log
I20260806 14:58:43.158658 21517 ts_tablet_manager.cc:1397] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 7887589cb7404641939ec7d93266fcca: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:43.158790 21517 raft_consensus.cc:348] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 7887589cb7404641939ec7d93266fcca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7887589cb7404641939ec7d93266fcca" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34831 } } peers { permanent_uuid: "cd4e5e034bb647b2adf0bbf9f4850843" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 46791 } } peers { permanent_uuid: "9e45adfc7114482c9bfd1cc0b6b75a96" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 40721 } }
I20260806 14:58:43.158859 21517 raft_consensus.cc:374] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 7887589cb7404641939ec7d93266fcca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:43.158883 21517 raft_consensus.cc:729] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 7887589cb7404641939ec7d93266fcca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7887589cb7404641939ec7d93266fcca, State: Initialized, Role: FOLLOWER
I20260806 14:58:43.158941 21517 consensus_queue.cc:260] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 7887589cb7404641939ec7d93266fcca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7887589cb7404641939ec7d93266fcca" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34831 } } peers { permanent_uuid: "cd4e5e034bb647b2adf0bbf9f4850843" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 46791 } } peers { permanent_uuid: "9e45adfc7114482c9bfd1cc0b6b75a96" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 40721 } }
I20260806 14:58:43.159109 21517 ts_tablet_manager.cc:1428] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 7887589cb7404641939ec7d93266fcca: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 14:58:43.159386 21483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.159291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.11.8.67:53979)
0806 14:58:43.159326 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:43.159354 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260806 14:58:43.159955 21365 heartbeater.cc:503] Master 127.11.8.125:42909 was elected leader, sending a full tablet report...
I20260806 14:58:43.160267 21483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.160078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.11.8.67:42181)
0806 14:58:43.160214 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:43.160244 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260806 14:58:43.162897 21512 tablet_bootstrap.cc:492] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 9e45adfc7114482c9bfd1cc0b6b75a96: No bootstrap required, opened a new log
I20260806 14:58:43.162966 21512 ts_tablet_manager.cc:1397] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 9e45adfc7114482c9bfd1cc0b6b75a96: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260806 14:58:43.163134 21512 raft_consensus.cc:348] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 9e45adfc7114482c9bfd1cc0b6b75a96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7887589cb7404641939ec7d93266fcca" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34831 } } peers { permanent_uuid: "cd4e5e034bb647b2adf0bbf9f4850843" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 46791 } } peers { permanent_uuid: "9e45adfc7114482c9bfd1cc0b6b75a96" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 40721 } }
I20260806 14:58:43.163221 21512 raft_consensus.cc:374] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 9e45adfc7114482c9bfd1cc0b6b75a96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:43.163244 21512 raft_consensus.cc:729] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 9e45adfc7114482c9bfd1cc0b6b75a96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e45adfc7114482c9bfd1cc0b6b75a96, State: Initialized, Role: FOLLOWER
I20260806 14:58:43.163298 21512 consensus_queue.cc:260] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 9e45adfc7114482c9bfd1cc0b6b75a96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7887589cb7404641939ec7d93266fcca" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34831 } } peers { permanent_uuid: "cd4e5e034bb647b2adf0bbf9f4850843" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 46791 } } peers { permanent_uuid: "9e45adfc7114482c9bfd1cc0b6b75a96" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 40721 } }
I20260806 14:58:43.163511 21512 ts_tablet_manager.cc:1428] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 9e45adfc7114482c9bfd1cc0b6b75a96: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:43.163820 21484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.163676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.11.8.65:45581)
0806 14:58:43.163755 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:43.163791 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 14:58:43.164402 21484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.164286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.11.8.65:33471)
0806 14:58:43.164321 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:43.164355 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260806 14:58:43.164454 20976 heartbeater.cc:503] Master 127.11.8.125:42909 was elected leader, sending a full tablet report...
I20260806 14:58:43.185653 21516 raft_consensus.cc:482] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 14:58:43.185775 21516 raft_consensus.cc:504] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7887589cb7404641939ec7d93266fcca" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34831 } } peers { permanent_uuid: "cd4e5e034bb647b2adf0bbf9f4850843" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 46791 } } peers { permanent_uuid: "9e45adfc7114482c9bfd1cc0b6b75a96" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 40721 } }
I20260806 14:58:43.186148 21516 leader_election.cc:288] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7887589cb7404641939ec7d93266fcca (127.11.8.67:34831), 9e45adfc7114482c9bfd1cc0b6b75a96 (127.11.8.65:40721)
I20260806 14:58:43.189841 21509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.186407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:34945 (local address 127.11.8.65:40721)
0806 14:58:43.186515 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:43.186517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:43.186944 (+ 427us) server_negotiation.cc:407] Connection header received
0806 14:58:43.187003 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:43.187004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:43.187015 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:43.187055 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:43.187193 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:43.187632 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:43.188938 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:43.189489 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:43.189569 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:43.189582 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:43.189737 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 14:58:43.189779 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 14:58:43.189841 21508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.186447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:39735 (local address 127.11.8.67:34831)
0806 14:58:43.186500 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:43.186502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:43.186519 (+ 17us) server_negotiation.cc:407] Connection header received
0806 14:58:43.186606 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:43.186607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:43.186617 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:43.186668 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:43.186889 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:43.187502 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:43.188938 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:43.189488 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:43.189576 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:43.189587 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:43.189737 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 14:58:43.189774 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 14:58:43.189882 21485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.186334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:34831 (local address 127.11.8.66:39735)
0806 14:58:43.186376 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:43.186388 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:43.186524 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:43.186701 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:43.186702 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:43.186705 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:43.186793 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:43.186796 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:43.187590 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:43.187591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:43.188836 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:43.188839 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:43.189585 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:43.189586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:43.189652 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:43.189654 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 14:58:43.189712 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 14:58:43.189759 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":15}
I20260806 14:58:43.189883 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.186334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:40721 (local address 127.11.8.66:34945)
0806 14:58:43.186796 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:43.186806 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:43.186944 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:43.187076 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:43.187076 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:43.187078 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:43.187132 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:43.187133 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:43.187745 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:43.187746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:43.188836 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:43.188839 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:43.189582 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:43.189583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:43.189648 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:43.189652 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 14:58:43.189712 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 14:58:43.189758 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":280,"thread_start_us":122,"threads_started":1}
I20260806 14:58:43.190224 20905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fbcc7daf7da4af0809d9e1a54c3cc26" candidate_uuid: "cd4e5e034bb647b2adf0bbf9f4850843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e45adfc7114482c9bfd1cc0b6b75a96" is_pre_election: true
I20260806 14:58:43.190212 21295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fbcc7daf7da4af0809d9e1a54c3cc26" candidate_uuid: "cd4e5e034bb647b2adf0bbf9f4850843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7887589cb7404641939ec7d93266fcca" is_pre_election: true
I20260806 14:58:43.190342 20905 raft_consensus.cc:2454] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 9e45adfc7114482c9bfd1cc0b6b75a96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd4e5e034bb647b2adf0bbf9f4850843 in term 0.
I20260806 14:58:43.190346 21295 raft_consensus.cc:2454] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 7887589cb7404641939ec7d93266fcca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd4e5e034bb647b2adf0bbf9f4850843 in term 0.
I20260806 14:58:43.190551 21000 leader_election.cc:302] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9e45adfc7114482c9bfd1cc0b6b75a96, cd4e5e034bb647b2adf0bbf9f4850843; no voters:
I20260806 14:58:43.190719 21516 raft_consensus.cc:2781] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 14:58:43.190774 21516 raft_consensus.cc:482] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 14:58:43.190797 21516 raft_consensus.cc:3037] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:43.191531 21516 raft_consensus.cc:504] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7887589cb7404641939ec7d93266fcca" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34831 } } peers { permanent_uuid: "cd4e5e034bb647b2adf0bbf9f4850843" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 46791 } } peers { permanent_uuid: "9e45adfc7114482c9bfd1cc0b6b75a96" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 40721 } }
I20260806 14:58:43.191706 21516 leader_election.cc:288] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843 [CANDIDATE]: Term 1 election: Requested vote from peers 7887589cb7404641939ec7d93266fcca (127.11.8.67:34831), 9e45adfc7114482c9bfd1cc0b6b75a96 (127.11.8.65:40721)
I20260806 14:58:43.191825 21295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fbcc7daf7da4af0809d9e1a54c3cc26" candidate_uuid: "cd4e5e034bb647b2adf0bbf9f4850843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7887589cb7404641939ec7d93266fcca"
I20260806 14:58:43.191905 21295 raft_consensus.cc:3037] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 7887589cb7404641939ec7d93266fcca [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:43.191939 20905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fbcc7daf7da4af0809d9e1a54c3cc26" candidate_uuid: "cd4e5e034bb647b2adf0bbf9f4850843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e45adfc7114482c9bfd1cc0b6b75a96"
I20260806 14:58:43.192049 20905 raft_consensus.cc:3037] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 9e45adfc7114482c9bfd1cc0b6b75a96 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:43.192601 21295 raft_consensus.cc:2454] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 7887589cb7404641939ec7d93266fcca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4e5e034bb647b2adf0bbf9f4850843 in term 1.
I20260806 14:58:43.192775 21006 leader_election.cc:302] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7887589cb7404641939ec7d93266fcca, cd4e5e034bb647b2adf0bbf9f4850843; no voters:
I20260806 14:58:43.192814 20905 raft_consensus.cc:2454] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 9e45adfc7114482c9bfd1cc0b6b75a96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4e5e034bb647b2adf0bbf9f4850843 in term 1.
I20260806 14:58:43.192896 21516 raft_consensus.cc:2781] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:43.193111 21516 raft_consensus.cc:686] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843 [term 1 LEADER]: Becoming Leader. State: Replica: cd4e5e034bb647b2adf0bbf9f4850843, State: Running, Role: LEADER
I20260806 14:58:43.193214 21516 consensus_queue.cc:237] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7887589cb7404641939ec7d93266fcca" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34831 } } peers { permanent_uuid: "cd4e5e034bb647b2adf0bbf9f4850843" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 46791 } } peers { permanent_uuid: "9e45adfc7114482c9bfd1cc0b6b75a96" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 40721 } }
I20260806 14:58:43.193444 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.193325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.11.8.66:45843)
0806 14:58:43.193358 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:43.193390 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260806 14:58:43.194142 20462 catalog_manager.cc:5495] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd4e5e034bb647b2adf0bbf9f4850843 (127.11.8.66). New cstate: current_term: 1 leader_uuid: "cd4e5e034bb647b2adf0bbf9f4850843" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7887589cb7404641939ec7d93266fcca" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd4e5e034bb647b2adf0bbf9f4850843" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 46791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9e45adfc7114482c9bfd1cc0b6b75a96" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 40721 } health_report { overall_health: UNKNOWN } } }
I20260806 14:58:43.201893 21506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.199277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49598 (local address 127.11.8.66:46791)
0806 14:58:43.199344 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:43.199346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:43.199355 (+ 9us) server_negotiation.cc:407] Connection header received
0806 14:58:43.199384 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:43.199385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:43.199403 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:43.199487 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 14:58:43.199606 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:43.200178 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:43.201052 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:43.201278 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:43.201365 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:43.201544 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 14:58:43.201676 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 14:58:43.201754 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:43.201800 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 14:58:43.201821 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 14:58:43.202982 20905 raft_consensus.cc:1261] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 9e45adfc7114482c9bfd1cc0b6b75a96 [term 1 FOLLOWER]: Refusing update from remote peer cd4e5e034bb647b2adf0bbf9f4850843: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:43.203024 21295 raft_consensus.cc:1261] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 7887589cb7404641939ec7d93266fcca [term 1 FOLLOWER]: Refusing update from remote peer cd4e5e034bb647b2adf0bbf9f4850843: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:43.203239 21516 consensus_queue.cc:1035] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e45adfc7114482c9bfd1cc0b6b75a96" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 40721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:43.203514 21516 consensus_queue.cc:1035] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7887589cb7404641939ec7d93266fcca" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:43.204552 21483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.204224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.11.8.67:46509)
0806 14:58:43.204468 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:43.204509 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228}
I20260806 14:58:43.204901 21484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.204762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.11.8.65:52277)
0806 14:58:43.204821 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:43.204858 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260806 14:58:43.205618 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.205517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.11.8.66:58847)
0806 14:58:43.205562 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:43.205591 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260806 14:58:43.206624 21541 mvcc.cc:205] Tried to move back new op lower bound from 7315572011837464576 to 7315572011800039424. Current Snapshot: MvccSnapshot[applied={T|T < 7315572011837464576 or (T in {7315572011837464576})}]
I20260806 14:58:43.207226 21483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.206567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.11.8.67:55353)
0806 14:58:43.207162 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:43.207193 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":577}
I20260806 14:58:43.207288 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.207190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.11.8.66:51137)
0806 14:58:43.207240 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:43.207262 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 14:58:43.207526 21484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.207408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34611 (local address 127.11.8.65:58945)
0806 14:58:43.207470 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:43.207494 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:34611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260806 14:58:43.207762 20462 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:58:43.207852 20462 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:58:43.209188 20462 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ef653e0dfa3841999c9d8f70b12f03e6: Sending DeleteTablet for 3 replicas of tablet 4fbcc7daf7da4af0809d9e1a54c3cc26
I20260806 14:58:43.209470 21265 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fbcc7daf7da4af0809d9e1a54c3cc26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:43 UTC) from {username='oryx', principal='oryx/127.11.8.125@KRBTEST.COM'} at 127.0.0.1:58030
I20260806 14:58:43.209684 21070 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fbcc7daf7da4af0809d9e1a54c3cc26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:43 UTC) from {username='oryx', principal='oryx/127.11.8.125@KRBTEST.COM'} at 127.0.0.1:49586
I20260806 14:58:43.209728 20878 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fbcc7daf7da4af0809d9e1a54c3cc26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:43 UTC) from {username='oryx', principal='oryx/127.11.8.125@KRBTEST.COM'} at 127.0.0.1:34324
I20260806 14:58:43.209931 21551 tablet_replica.cc:330] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 9e45adfc7114482c9bfd1cc0b6b75a96: stopping tablet replica
I20260806 14:58:43.209964 21549 tablet_replica.cc:330] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 7887589cb7404641939ec7d93266fcca: stopping tablet replica
I20260806 14:58:43.210026 21549 raft_consensus.cc:2229] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 7887589cb7404641939ec7d93266fcca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:58:43.210078 21549 raft_consensus.cc:2258] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 7887589cb7404641939ec7d93266fcca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:43.210065 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 20789
I20260806 14:58:43.212538 21549 ts_tablet_manager.cc:1905] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 7887589cb7404641939ec7d93266fcca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:43.214876 21549 ts_tablet_manager.cc:1918] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 7887589cb7404641939ec7d93266fcca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 14:58:43.214953 21549 log.cc:1196] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 7887589cb7404641939ec7d93266fcca: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-2/wal/wals/4fbcc7daf7da4af0809d9e1a54c3cc26
I20260806 14:58:43.215253 21549 ts_tablet_manager.cc:1939] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P 7887589cb7404641939ec7d93266fcca: Deleting consensus metadata
I20260806 14:58:43.215665 20444 catalog_manager.cc:4841] TS 7887589cb7404641939ec7d93266fcca (127.11.8.67:34831): tablet 4fbcc7daf7da4af0809d9e1a54c3cc26 (table test-table [id=c02118425c8d4812b3b43eb1c0e42293]) successfully deleted
I20260806 14:58:43.218549 21550 tablet_replica.cc:330] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843: stopping tablet replica
I20260806 14:58:43.218674 21550 raft_consensus.cc:2229] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843 [term 1 LEADER]: Raft consensus shutting down.
I20260806 14:58:43.218825 21550 raft_consensus.cc:2258] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:43.219486 21550 ts_tablet_manager.cc:1905] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:43.221101 21550 ts_tablet_manager.cc:1918] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 14:58:43.221211 21550 log.cc:1196] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786028287744255-11297-0/minicluster-data/ts-1/wal/wals/4fbcc7daf7da4af0809d9e1a54c3cc26
I20260806 14:58:43.221521 21550 ts_tablet_manager.cc:1939] T 4fbcc7daf7da4af0809d9e1a54c3cc26 P cd4e5e034bb647b2adf0bbf9f4850843: Deleting consensus metadata
I20260806 14:58:43.221939 20442 catalog_manager.cc:4841] TS cd4e5e034bb647b2adf0bbf9f4850843 (127.11.8.66:46791): tablet 4fbcc7daf7da4af0809d9e1a54c3cc26 (table test-table [id=c02118425c8d4812b3b43eb1c0e42293]) successfully deleted
I20260806 14:58:43.226347 21507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:43.226204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:40721 (local address 127.0.0.1:34334)
0806 14:58:43.226275 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:43.226307 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.65:40721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 14:58:43.226531 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 20982
W20260806 14:58:43.226576 20442 catalog_manager.cc:4584] TS 9e45adfc7114482c9bfd1cc0b6b75a96 (127.11.8.65:40721): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4fbcc7daf7da4af0809d9e1a54c3cc26: Network error: Client connection negotiation failed: client connection to 127.11.8.65:40721: connect: Connection refused (error 111)
I20260806 14:58:43.245582 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 21171
I20260806 14:58:43.257148 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 20264
I20260806 14:58:43.266845 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 20534
2026-08-06T14:58:43Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6939 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 14:58:43 dist-test-slave-68xk krb5kdc[21554](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 14:58:43 dist-test-slave-68xk krb5kdc[21554](info): set up 2 sockets
Aug 06 14:58:43 dist-test-slave-68xk krb5kdc[21554](info): commencing operation
krb5kdc: starting...
W20260806 14:58:44.483027 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 14:58:44 dist-test-slave-68xk krb5kdc[21554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T14:58:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T14:58:44Z Disabled control of system clock
WARNING: no policy specified for oryx/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.126.keytab.
Entry for principal oryx/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.126.keytab.
I20260806 14:58:44.606091 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:36493
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.126.keytab
--principal=oryx/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:39667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:36493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:44.689582 21571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:44.689791 21571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:44.689814 21571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:44.690971 21571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 14:58:44.691010 21571 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 14:58:44.691025 21571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:44.691037 21571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:44.691051 21571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 14:58:44.691062 21571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 14:58:44.693245 21571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:39667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:36493
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.126.keytab
--principal=oryx/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:36493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:44.693495 21571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:44.693733 21571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:44.698707 21579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:44.698740 21578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:44.698712 21581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:44.698945 21571 server_base.cc:1034] running on GCE node
I20260806 14:58:44.699270 21571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:44.699563 21571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:44.700740 21571 hybrid_clock.cc:648] HybridClock initialized: now 1786028324700724 us; error 28 us; skew 500 ppm
Aug 06 14:58:44 dist-test-slave-68xk krb5kdc[21554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028324, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:44.703249 21571 init.cc:377] Logged in from keytab as oryx/127.11.8.126@KRBTEST.COM (short username oryx)
I20260806 14:58:44.703790 21571 webserver.cc:460] Webserver started at http://127.11.8.126:40481/ using document root <none> and password file <none>
I20260806 14:58:44.703989 21571 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:44.704030 21571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:44.704123 21571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:44.705124 21571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "b79710c047b8421f88b764f7af7025c6"
format_stamp: "Formatted at 2026-08-06 14:58:44 on dist-test-slave-68xk"
server_key: "128916a0955cc9ed9b126850eebe9569"
server_key_iv: "c8bb2b5354ed147e6960dfe0ebacf6ce"
server_key_version: "encryptionkey@0"
I20260806 14:58:44.705425 21571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "b79710c047b8421f88b764f7af7025c6"
format_stamp: "Formatted at 2026-08-06 14:58:44 on dist-test-slave-68xk"
server_key: "128916a0955cc9ed9b126850eebe9569"
server_key_iv: "c8bb2b5354ed147e6960dfe0ebacf6ce"
server_key_version: "encryptionkey@0"
I20260806 14:58:44.706763 21571 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 14:58:44.707490 21588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:44.707679 21571 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 14:58:44.707767 21571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "b79710c047b8421f88b764f7af7025c6"
format_stamp: "Formatted at 2026-08-06 14:58:44 on dist-test-slave-68xk"
server_key: "128916a0955cc9ed9b126850eebe9569"
server_key_iv: "c8bb2b5354ed147e6960dfe0ebacf6ce"
server_key_version: "encryptionkey@0"
I20260806 14:58:44.707820 21571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:44.726023 21571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:44.728171 21571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:44.728312 21571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:44.732823 21571 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:36493
I20260806 14:58:44.732966 21678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:36493 every 8 connection(s)
I20260806 14:58:44.733271 21571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 14:58:44.734038 21679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:44.736398 21679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b79710c047b8421f88b764f7af7025c6: Bootstrap starting.
I20260806 14:58:44.737037 21679 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b79710c047b8421f88b764f7af7025c6: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:44.737344 21679 log.cc:824] T 00000000000000000000000000000000 P b79710c047b8421f88b764f7af7025c6: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:44.738065 21679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b79710c047b8421f88b764f7af7025c6: No bootstrap required, opened a new log
I20260806 14:58:44.740173 21679 raft_consensus.cc:348] T 00000000000000000000000000000000 P b79710c047b8421f88b764f7af7025c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b79710c047b8421f88b764f7af7025c6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 36493 } }
I20260806 14:58:44.740346 21679 raft_consensus.cc:374] T 00000000000000000000000000000000 P b79710c047b8421f88b764f7af7025c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:44.740398 21679 raft_consensus.cc:729] T 00000000000000000000000000000000 P b79710c047b8421f88b764f7af7025c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b79710c047b8421f88b764f7af7025c6, State: Initialized, Role: FOLLOWER
I20260806 14:58:44.740553 21679 consensus_queue.cc:260] T 00000000000000000000000000000000 P b79710c047b8421f88b764f7af7025c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b79710c047b8421f88b764f7af7025c6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 36493 } }
I20260806 14:58:44.740638 21679 raft_consensus.cc:388] T 00000000000000000000000000000000 P b79710c047b8421f88b764f7af7025c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 14:58:44.740682 21679 raft_consensus.cc:482] T 00000000000000000000000000000000 P b79710c047b8421f88b764f7af7025c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 14:58:44.740751 21679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b79710c047b8421f88b764f7af7025c6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:44.740952 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 21571
I20260806 14:58:44.741057 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 14:58:44.741250 11297 external_mini_cluster.cc:1428] Setting key 38a33c8abf76e3c7b138427ac494bf43
I20260806 14:58:44.741438 21679 raft_consensus.cc:504] T 00000000000000000000000000000000 P b79710c047b8421f88b764f7af7025c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b79710c047b8421f88b764f7af7025c6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 36493 } }
I20260806 14:58:44.741554 21679 leader_election.cc:302] T 00000000000000000000000000000000 P b79710c047b8421f88b764f7af7025c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b79710c047b8421f88b764f7af7025c6; no voters:
I20260806 14:58:44.741751 21679 leader_election.cc:288] T 00000000000000000000000000000000 P b79710c047b8421f88b764f7af7025c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 14:58:44.741782 21682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b79710c047b8421f88b764f7af7025c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:44.742008 21682 raft_consensus.cc:686] T 00000000000000000000000000000000 P b79710c047b8421f88b764f7af7025c6 [term 1 LEADER]: Becoming Leader. State: Replica: b79710c047b8421f88b764f7af7025c6, State: Running, Role: LEADER
I20260806 14:58:44.742041 21679 sys_catalog.cc:564] T 00000000000000000000000000000000 P b79710c047b8421f88b764f7af7025c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 14:58:44.742254 21682 consensus_queue.cc:237] T 00000000000000000000000000000000 P b79710c047b8421f88b764f7af7025c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b79710c047b8421f88b764f7af7025c6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 36493 } }
I20260806 14:58:44.742702 21686 sys_catalog.cc:455] T 00000000000000000000000000000000 P b79710c047b8421f88b764f7af7025c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b79710c047b8421f88b764f7af7025c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b79710c047b8421f88b764f7af7025c6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 36493 } } }
I20260806 14:58:44.742794 21686 sys_catalog.cc:458] T 00000000000000000000000000000000 P b79710c047b8421f88b764f7af7025c6 [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:44.742690 21687 sys_catalog.cc:455] T 00000000000000000000000000000000 P b79710c047b8421f88b764f7af7025c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b79710c047b8421f88b764f7af7025c6. Latest consensus state: current_term: 1 leader_uuid: "b79710c047b8421f88b764f7af7025c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b79710c047b8421f88b764f7af7025c6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 36493 } } }
I20260806 14:58:44.743012 21687 sys_catalog.cc:458] T 00000000000000000000000000000000 P b79710c047b8421f88b764f7af7025c6 [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:44.743428 21696 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 14:58:44.743996 21696 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 14:58:44.745658 21696 catalog_manager.cc:1347] Generated new cluster ID: 58387808e0404e4889bb1207d19fa0ae
I20260806 14:58:44.745716 21696 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 14:58:44.750936 21696 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 14:58:44.751487 21696 catalog_manager.cc:1504] Loading token signing keys...
Aug 06 14:58:44 dist-test-slave-68xk krb5kdc[21554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM
I20260806 14:58:44.758577 21685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:44.741788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38280 (local address 127.11.8.126:36493)
0806 14:58:44.741976 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:44.741987 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:44.742472 (+ 485us) server_negotiation.cc:407] Connection header received
0806 14:58:44.742601 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:44.742610 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:44.742678 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:44.742844 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:44.743835 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:44.745281 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:44.751201 (+ 5920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:44.751879 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:44.751997 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:44.754935 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:44.754940 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:44.754941 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:44.754953 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:44.756892 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:44.757305 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:44.757308 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:44.757309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:44.757409 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:44.757617 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:44.757618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:44.757618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:44.757890 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:44.758028 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:44.758139 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 14:58:44.758297 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":87,"threads_started":1}
I20260806 14:58:44.759188 21696 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b79710c047b8421f88b764f7af7025c6: Generated new TSK 0
I20260806 14:58:44.759418 21696 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.11.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.65.keytab.
Entry for principal oryx/127.11.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.65.keytab.
I20260806 14:58:44.797570 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:0
--local_ip_for_outbound_sockets=127.11.8.65
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:36493
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.65.keytab
--principal=oryx/127.11.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:39667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:44.895239 21712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:44.895445 21712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:44.895473 21712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:44.896752 21712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:44.896795 21712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:44.896814 21712 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 14:58:44.896832 21712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 14:58:44.898357 21712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:39667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.65.keytab
--principal=oryx/127.11.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.8.126:36493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:44.898633 21712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:44.898888 21712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:44.903995 21721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:44.904000 21723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:44.904043 21725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:44.904366 21712 server_base.cc:1034] running on GCE node
I20260806 14:58:44.904582 21712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:44.904819 21712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:44.906018 21712 hybrid_clock.cc:648] HybridClock initialized: now 1786028324905977 us; error 54 us; skew 500 ppm
Aug 06 14:58:44 dist-test-slave-68xk krb5kdc[21554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028324, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:44.909010 21712 init.cc:377] Logged in from keytab as oryx/127.11.8.65@KRBTEST.COM (short username oryx)
I20260806 14:58:44.909552 21712 webserver.cc:460] Webserver started at http://127.11.8.65:38727/ using document root <none> and password file <none>
I20260806 14:58:44.909806 21712 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:44.909883 21712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:44.910068 21712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:44.911329 21712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-0/data/instance:
uuid: "4c9b8da547274fca8cbca18abfebc2d0"
format_stamp: "Formatted at 2026-08-06 14:58:44 on dist-test-slave-68xk"
server_key: "f425f43d3ac7c92f946d36736796aec3"
server_key_iv: "eb55c38a399cb3ac0d5a44183853b552"
server_key_version: "encryptionkey@0"
I20260806 14:58:44.911743 21712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance:
uuid: "4c9b8da547274fca8cbca18abfebc2d0"
format_stamp: "Formatted at 2026-08-06 14:58:44 on dist-test-slave-68xk"
server_key: "f425f43d3ac7c92f946d36736796aec3"
server_key_iv: "eb55c38a399cb3ac0d5a44183853b552"
server_key_version: "encryptionkey@0"
I20260806 14:58:44.913224 21712 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 14:58:44.914065 21737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:44.914244 21712 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 14:58:44.914364 21712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "4c9b8da547274fca8cbca18abfebc2d0"
format_stamp: "Formatted at 2026-08-06 14:58:44 on dist-test-slave-68xk"
server_key: "f425f43d3ac7c92f946d36736796aec3"
server_key_iv: "eb55c38a399cb3ac0d5a44183853b552"
server_key_version: "encryptionkey@0"
I20260806 14:58:44.914472 21712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:44.931879 21712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:44.934790 21712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:44.934986 21712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:44.935590 21712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:44.935663 21712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:44.935719 21712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:44.935752 21712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 14:58:44 dist-test-slave-68xk krb5kdc[21554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028324, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM
I20260806 14:58:44.943826 21712 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:34101
I20260806 14:58:44.943925 21901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:34101 every 8 connection(s)
I20260806 14:58:44.944296 21712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
I20260806 14:58:44.947547 21685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:44.936205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:35653 (local address 127.11.8.126:36493)
0806 14:58:44.936288 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:44.936291 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:44.936452 (+ 161us) server_negotiation.cc:407] Connection header received
0806 14:58:44.936962 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:44.936964 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:44.936982 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:44.937072 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:44.937993 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:44.938989 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:44.940663 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:44.941069 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:44.941154 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:44.944439 (+ 3285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:44.944441 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:44.944442 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:44.944451 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:44.946301 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:44.946701 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:44.946702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:44.946703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:44.946761 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:44.946987 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:44.946988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:44.946988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:44.947099 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:44.947157 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:44.947382 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 14:58:44.947459 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 14:58:44.947626 21748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:44.936121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:36493 (local address 127.11.8.65:35653)
0806 14:58:44.936309 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:44.936351 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:44.936855 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:44.937165 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:44.937169 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:44.937560 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:44.937865 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:44.937873 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:44.939106 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:44.939108 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:44.940552 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:44.940557 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:44.941196 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:44.941197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:44.941337 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:44.941946 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:44.941955 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:44.944315 (+ 2360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:44.946382 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:44.946388 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:44.946390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:44.946628 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:44.946825 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:44.946826 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:44.946826 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:44.946928 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:44.947158 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:44.947163 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:44.947276 (+ 113us) client_negotiation.cc:769] Sending connection context
0806 14:58:44.947357 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 14:58:44.947477 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":105,"threads_started":1}
I20260806 14:58:44.948359 21902 heartbeater.cc:347] Connected to a master server at 127.11.8.126:36493
I20260806 14:58:44.948473 21902 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:44.948690 21902 heartbeater.cc:511] Master 127.11.8.126:36493 requested a full tablet report, sending...
I20260806 14:58:44.949319 21613 ts_manager.cc:194] Registered new tserver with Master: 4c9b8da547274fca8cbca18abfebc2d0 (127.11.8.65:34101)
I20260806 14:58:44.950269 21613 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.8.65@KRBTEST.COM'} at 127.11.8.65:35653
I20260806 14:58:44.954144 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 21712
I20260806 14:58:44.954280 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance
I20260806 14:58:44.954532 11297 external_mini_cluster.cc:1428] Setting key de0fde1710ede305be471c594dbc84e9
I20260806 14:58:44.955621 21685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:44.949452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:37795 (local address 127.11.8.126:36493)
0806 14:58:44.949492 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:44.949494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:44.949595 (+ 101us) server_negotiation.cc:407] Connection header received
0806 14:58:44.949619 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:44.949620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:44.949630 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:44.949703 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:44.949979 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:44.950688 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:44.951549 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:44.951748 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:44.951810 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:44.952633 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:44.952635 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:44.952635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:44.952643 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:44.954238 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:44.954749 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:44.954754 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:44.954755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:44.954829 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:44.955095 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:44.955096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:44.955096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:44.955193 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:44.955247 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:44.955437 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 14:58:44.955536 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 14:58:44.955652 21911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:44.949373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:36493 (local address 127.11.8.65:37795)
0806 14:58:44.949514 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:44.949524 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:44.949583 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:44.949705 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:44.949705 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:44.949854 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:44.949909 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:44.949911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:44.950761 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:44.950762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:44.951437 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:44.951439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:44.951819 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:44.951820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:44.951871 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:44.952265 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:44.952269 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:44.952551 (+ 282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:44.954360 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:44.954362 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:44.954362 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:44.954606 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:44.954927 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:44.954928 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:44.954928 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:44.955012 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:44.955259 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:44.955260 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:44.955323 (+ 63us) client_negotiation.cc:769] Sending connection context
0806 14:58:44.955401 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 14:58:44.955524 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
WARNING: no policy specified for oryx/127.11.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.66.keytab.
Entry for principal oryx/127.11.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.66.keytab.
I20260806 14:58:44.983206 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:0
--local_ip_for_outbound_sockets=127.11.8.66
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:36493
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.66.keytab
--principal=oryx/127.11.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:39667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:45.068617 21918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:45.068812 21918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:45.068835 21918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:45.069903 21918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:45.069938 21918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:45.069957 21918 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 14:58:45.069974 21918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 14:58:45.071419 21918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:39667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.66.keytab
--principal=oryx/127.11.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.8.126:36493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:45.071595 21918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:45.071758 21918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:45.076782 21930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:45.076757 21924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:45.076776 21926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:45.076930 21918 server_base.cc:1034] running on GCE node
I20260806 14:58:45.077136 21918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:45.077378 21918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:45.078550 21918 hybrid_clock.cc:648] HybridClock initialized: now 1786028325078541 us; error 43 us; skew 500 ppm
Aug 06 14:58:45 dist-test-slave-68xk krb5kdc[21554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028325, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:45.081012 21918 init.cc:377] Logged in from keytab as oryx/127.11.8.66@KRBTEST.COM (short username oryx)
I20260806 14:58:45.081516 21918 webserver.cc:460] Webserver started at http://127.11.8.66:38617/ using document root <none> and password file <none>
I20260806 14:58:45.081743 21918 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:45.081809 21918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:45.081941 21918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:45.083012 21918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-1/data/instance:
uuid: "e2751e00e4174c9ebcf59dd0ea26cb94"
format_stamp: "Formatted at 2026-08-06 14:58:45 on dist-test-slave-68xk"
server_key: "409d1f09e2b78b89356e2710482a3e5e"
server_key_iv: "16aea71947e027fdf82a2457fa9f987b"
server_key_version: "encryptionkey@0"
I20260806 14:58:45.083379 21918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance:
uuid: "e2751e00e4174c9ebcf59dd0ea26cb94"
format_stamp: "Formatted at 2026-08-06 14:58:45 on dist-test-slave-68xk"
server_key: "409d1f09e2b78b89356e2710482a3e5e"
server_key_iv: "16aea71947e027fdf82a2457fa9f987b"
server_key_version: "encryptionkey@0"
I20260806 14:58:45.084757 21918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 14:58:45.085563 21937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:45.085757 21918 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:45.085856 21918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "e2751e00e4174c9ebcf59dd0ea26cb94"
format_stamp: "Formatted at 2026-08-06 14:58:45 on dist-test-slave-68xk"
server_key: "409d1f09e2b78b89356e2710482a3e5e"
server_key_iv: "16aea71947e027fdf82a2457fa9f987b"
server_key_version: "encryptionkey@0"
I20260806 14:58:45.085935 21918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:45.105422 21918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:45.107755 21918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:45.107914 21918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:45.108647 21918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:45.108707 21918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:45.108752 21918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:45.108789 21918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 14:58:45 dist-test-slave-68xk krb5kdc[21554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028325, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM
I20260806 14:58:45.116734 21918 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:36205
I20260806 14:58:45.117161 21918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
I20260806 14:58:45.117964 22109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:36205 every 8 connection(s)
I20260806 14:58:45.118707 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 21918
I20260806 14:58:45.118813 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance
I20260806 14:58:45.119042 11297 external_mini_cluster.cc:1428] Setting key 6ab73523c89da1a31f440d3a62001474
I20260806 14:58:45.119108 21685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.109042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:40733 (local address 127.11.8.126:36493)
0806 14:58:45.109115 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:45.109117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:45.109344 (+ 227us) server_negotiation.cc:407] Connection header received
0806 14:58:45.109878 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:45.109880 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:45.109893 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:45.109949 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:45.110688 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.111311 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.112348 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.112590 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.112679 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.115664 (+ 2985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:45.115666 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:45.115667 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:45.115677 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:45.117522 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:45.117964 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:45.117965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:45.117965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:45.118022 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:45.118319 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:45.118320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:45.118320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:45.118490 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:45.118598 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:45.118904 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 14:58:45.119006 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 14:58:45.119223 21948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.109037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:36493 (local address 127.11.8.66:40733)
0806 14:58:45.109212 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:45.109251 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:45.109725 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:45.110039 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:45.110043 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:45.110305 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:45.110585 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:45.110590 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.111398 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.111399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:45.112252 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:45.112254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.112689 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.112690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:45.112818 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.113326 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:45.113337 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:45.115585 (+ 2248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:45.117659 (+ 2074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:45.117666 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:45.117668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:45.117893 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:45.118106 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:45.118107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:45.118107 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:45.118244 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:45.118592 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:45.118600 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:45.118734 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 14:58:45.118847 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 14:58:45.119005 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":85,"threads_started":1}
I20260806 14:58:45.119705 22110 heartbeater.cc:347] Connected to a master server at 127.11.8.126:36493
I20260806 14:58:45.119792 22110 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:45.119974 22110 heartbeater.cc:511] Master 127.11.8.126:36493 requested a full tablet report, sending...
I20260806 14:58:45.121673 21613 ts_manager.cc:194] Registered new tserver with Master: e2751e00e4174c9ebcf59dd0ea26cb94 (127.11.8.66:36205)
I20260806 14:58:45.122035 21613 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.8.66@KRBTEST.COM'} at 127.11.8.66:40733
I20260806 14:58:45.127640 22119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.120664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:36493 (local address 127.11.8.66:58257)
0806 14:58:45.120801 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:45.120811 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:45.120862 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:45.121777 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:45.121778 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:45.121983 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:45.122056 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:45.122058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.122976 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.122977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:45.123504 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:45.123507 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.123995 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.123996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:45.124043 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.124474 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:45.124478 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:45.124767 (+ 289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:45.126637 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:45.126638 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:45.126638 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:45.126807 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:45.127021 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:45.127022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:45.127022 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:45.127070 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:45.127405 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:45.127406 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:45.127450 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 14:58:45.127501 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 14:58:45.127574 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260806 14:58:45.127753 21685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.120739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:58257 (local address 127.11.8.126:36493)
0806 14:58:45.121640 (+ 901us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:45.121642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:45.121650 (+ 8us) server_negotiation.cc:407] Connection header received
0806 14:58:45.121667 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:45.121668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:45.121677 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:45.121735 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:45.122149 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.122869 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.123599 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.123914 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.123992 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.124849 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:45.124851 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:45.124851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:45.124859 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:45.126545 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:45.126880 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:45.126882 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:45.126882 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:45.126950 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:45.127136 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:45.127137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:45.127138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:45.127334 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:45.127412 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:45.127551 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 14:58:45.127642 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":883}
WARNING: no policy specified for oryx/127.11.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.67.keytab.
Entry for principal oryx/127.11.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.67.keytab.
I20260806 14:58:45.149235 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:0
--local_ip_for_outbound_sockets=127.11.8.67
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:36493
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.67.keytab
--principal=oryx/127.11.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:39667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:45.234256 22127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:45.234510 22127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:45.234539 22127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:45.235731 22127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:45.235773 22127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:45.235795 22127 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 14:58:45.235843 22127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 14:58:45.237355 22127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:39667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.67.keytab
--principal=oryx/127.11.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.11.8.126:36493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:45.237607 22127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:45.237864 22127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:45.242957 22136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:45.243152 22127 server_base.cc:1034] running on GCE node
W20260806 14:58:45.243005 22134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:45.243373 22139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:45.243568 22127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:45.243857 22127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:45.245049 22127 hybrid_clock.cc:648] HybridClock initialized: now 1786028325245025 us; error 41 us; skew 500 ppm
Aug 06 14:58:45 dist-test-slave-68xk krb5kdc[21554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028325, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:45.247640 22127 init.cc:377] Logged in from keytab as oryx/127.11.8.67@KRBTEST.COM (short username oryx)
I20260806 14:58:45.248068 22127 webserver.cc:460] Webserver started at http://127.11.8.67:34775/ using document root <none> and password file <none>
I20260806 14:58:45.248322 22127 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:45.248368 22127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:45.248514 22127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:45.249625 22127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-2/data/instance:
uuid: "37ad3fe8e68b4d488ef0b44d31584401"
format_stamp: "Formatted at 2026-08-06 14:58:45 on dist-test-slave-68xk"
server_key: "e5d1f57f541cf67727d6c1a1b0b5957a"
server_key_iv: "8f67b3654fab2ad00e0adaee7d19d404"
server_key_version: "encryptionkey@0"
I20260806 14:58:45.249986 22127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance:
uuid: "37ad3fe8e68b4d488ef0b44d31584401"
format_stamp: "Formatted at 2026-08-06 14:58:45 on dist-test-slave-68xk"
server_key: "e5d1f57f541cf67727d6c1a1b0b5957a"
server_key_iv: "8f67b3654fab2ad00e0adaee7d19d404"
server_key_version: "encryptionkey@0"
I20260806 14:58:45.251410 22127 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 14:58:45.252213 22152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:45.252480 22127 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 14:58:45.252578 22127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "37ad3fe8e68b4d488ef0b44d31584401"
format_stamp: "Formatted at 2026-08-06 14:58:45 on dist-test-slave-68xk"
server_key: "e5d1f57f541cf67727d6c1a1b0b5957a"
server_key_iv: "8f67b3654fab2ad00e0adaee7d19d404"
server_key_version: "encryptionkey@0"
I20260806 14:58:45.252651 22127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:45.276036 22127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:45.278559 22127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:45.278728 22127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:45.279259 22127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:45.279328 22127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:45.279379 22127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:45.279425 22127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 14:58:45 dist-test-slave-68xk krb5kdc[21554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028325, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM
I20260806 14:58:45.286955 22127 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:40909
I20260806 14:58:45.287016 22316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:40909 every 8 connection(s)
I20260806 14:58:45.287374 22127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
I20260806 14:58:45.289639 21685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.279864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:56633 (local address 127.11.8.126:36493)
0806 14:58:45.279944 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:45.279946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:45.280180 (+ 234us) server_negotiation.cc:407] Connection header received
0806 14:58:45.280941 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:45.280942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:45.280953 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:45.281065 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:45.281994 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.282893 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.283940 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.284157 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.284247 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.286830 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:45.286832 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:45.286832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:45.286843 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:45.288465 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:45.288892 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:45.288894 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:45.288894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:45.288944 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:45.289129 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:45.289130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:45.289130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:45.289231 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:45.289283 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:45.289512 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 14:58:45.289578 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 14:58:45.289845 22163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.279823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:36493 (local address 127.11.8.67:56633)
0806 14:58:45.280029 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:45.280079 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:45.280809 (+ 730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:45.281171 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:45.281177 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:45.281600 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:45.281908 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:45.281914 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.282994 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.282995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:45.283853 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:45.283856 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.284273 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.284275 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:45.284405 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.284986 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:45.284996 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:45.286725 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:45.288581 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:45.288588 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:45.288590 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:45.288826 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:45.288987 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:45.288987 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:45.288988 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:45.289072 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:45.289289 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:45.289293 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:45.289405 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 14:58:45.289485 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 14:58:45.289603 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":93,"threads_started":1}
I20260806 14:58:45.290355 22317 heartbeater.cc:347] Connected to a master server at 127.11.8.126:36493
I20260806 14:58:45.290474 22317 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:45.290652 22317 heartbeater.cc:511] Master 127.11.8.126:36493 requested a full tablet report, sending...
I20260806 14:58:45.291085 21611 ts_manager.cc:194] Registered new tserver with Master: 37ad3fe8e68b4d488ef0b44d31584401 (127.11.8.67:40909)
I20260806 14:58:45.291469 21611 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.11.8.67@KRBTEST.COM'} at 127.11.8.67:56633
I20260806 14:58:45.294783 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 22127
I20260806 14:58:45.294893 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance
I20260806 14:58:45.295074 11297 external_mini_cluster.cc:1428] Setting key cffbdf557e36dc5d0dfceb8b9a9fbf50
I20260806 14:58:45.295967 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 14:58:45.297219 22327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.291478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:36493 (local address 127.11.8.67:43667)
0806 14:58:45.291615 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:45.291625 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:45.291680 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:45.291806 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:45.291807 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:45.291985 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:45.292073 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:45.292075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.292686 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.292687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:45.293239 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:45.293240 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.293630 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.293631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:45.293695 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.294056 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:45.294061 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:45.294330 (+ 269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:45.296289 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:45.296290 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:45.296291 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:45.296458 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:45.296668 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:45.296668 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:45.296669 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:45.296719 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:45.296989 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:45.296989 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:45.297028 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 14:58:45.297080 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 14:58:45.297155 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
I20260806 14:58:45.297358 21685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.291536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:43667 (local address 127.11.8.126:36493)
0806 14:58:45.291613 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:45.291615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:45.291666 (+ 51us) server_negotiation.cc:407] Connection header received
0806 14:58:45.291736 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:45.291737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:45.291747 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:45.291801 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:45.292143 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.292595 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.293334 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.293542 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.293626 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.294450 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:45.294451 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:45.294452 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:45.294459 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:45.296195 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:45.296538 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:45.296539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:45.296539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:45.296604 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:45.296780 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:45.296780 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:45.296781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:45.296918 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:45.296987 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:45.297130 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 14:58:45.297214 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 14:58:45 dist-test-slave-68xk krb5kdc[21554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028325, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 14:58:45 dist-test-slave-68xk krb5kdc[21554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:45 dist-test-slave-68xk krb5kdc[21554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 14:58:45.319108 21685 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:45.313539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38288 (local address 127.11.8.126:36493)
0806 14:58:45.313600 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:45.313602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:45.313611 (+ 9us) server_negotiation.cc:407] Connection header received
0806 14:58:45.313710 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:45.313711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:45.313728 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:45.313798 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:45.314362 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.314915 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.315665 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.315859 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.315938 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.319011 (+ 3073us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38288: BlockingRecv error: recv got EOF from 127.0.0.1:38288 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 06 14:58:45 dist-test-slave-68xk krb5kdc[21554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028325, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM
I20260806 14:58:45.326261 21685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.319932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38302 (local address 127.11.8.126:36493)
0806 14:58:45.320000 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:45.320002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:45.320010 (+ 8us) server_negotiation.cc:407] Connection header received
0806 14:58:45.320061 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:45.320062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:45.320070 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:45.320127 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:45.320557 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.321076 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.321714 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.321889 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.321945 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.323927 (+ 1982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:45.323928 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:45.323928 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:45.323935 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:45.325350 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:45.325660 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:45.325661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:45.325662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:45.325718 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:45.325891 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:45.325892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:45.325892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:45.325990 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:45.326064 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:45.326142 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 14:58:45.326202 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 14:58:45.327955 21611 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38302:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 14:58:45.329542 21611 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 14:58:45.335852 22354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.331875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:34101 (local address 127.0.0.1:39080)
0806 14:58:45.332182 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:45.332191 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:45.332232 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:45.332570 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:45.332570 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:45.332572 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:45.332609 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:45.332610 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.333499 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.333500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:45.334585 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:45.334588 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.335498 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.335499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:45.335588 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.335606 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 14:58:45.335709 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 14:58:45.335766 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":4,"thread_start_us":55,"threads_started":1}
I20260806 14:58:45.335879 22356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.331996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39080 (local address 127.11.8.65:34101)
0806 14:58:45.332404 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:45.332408 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:45.332417 (+ 9us) server_negotiation.cc:407] Connection header received
0806 14:58:45.332452 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:45.332454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:45.332470 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:45.332573 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:45.332668 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.333404 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.334686 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.335413 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.335497 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.335587 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:45.335729 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 14:58:45.335778 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":82,"threads_started":1}
I20260806 14:58:45.336429 22357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.332071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:36205 (local address 127.0.0.1:58284)
0806 14:58:45.332248 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:45.332256 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:45.332291 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:45.332712 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:45.332713 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:45.332715 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:45.332750 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:45.332751 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.334052 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.334053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:45.335038 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:45.335041 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.336160 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.336161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:45.336229 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.336233 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 14:58:45.336323 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 14:58:45.336362 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":28,"thread_start_us":66,"threads_started":1}
I20260806 14:58:45.336576 22361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.332356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58284 (local address 127.11.8.66:36205)
0806 14:58:45.332546 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:45.332551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:45.332561 (+ 10us) server_negotiation.cc:407] Connection header received
0806 14:58:45.332600 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:45.332622 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:45.332639 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:45.332719 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:45.332853 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.333948 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.335129 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.335884 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.336119 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.336203 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:45.336342 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 14:58:45.336382 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":82,"threads_started":1}
I20260806 14:58:45.336688 21810 tablet_service.cc:1467] Processing CreateTablet for tablet 61e212624536415faca745eb05ea9345 (DEFAULT_TABLE table=test-table [id=ac04ba35713b4338a97c422e925d8def]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:45.337003 22355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.331875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:40909 (local address 127.0.0.1:42396)
0806 14:58:45.332044 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:45.332083 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:45.332145 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:45.332398 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:45.332402 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:45.332404 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:45.332469 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:45.332471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.334055 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.334056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:45.335215 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:45.335217 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.336569 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.336570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:45.336839 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.336845 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 14:58:45.336879 (+ 34us) client_negotiation.cc:240] Negotiation successful
0806 14:58:45.336924 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":25,"thread_start_us":71,"threads_started":1}
I20260806 14:58:45.337116 21810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61e212624536415faca745eb05ea9345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:45.337306 22358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.332070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42396 (local address 127.11.8.67:40909)
0806 14:58:45.332223 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:45.332228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:45.332240 (+ 12us) server_negotiation.cc:407] Connection header received
0806 14:58:45.332280 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:45.332283 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:45.332301 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:45.332398 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:45.332830 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.333944 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.335318 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.336080 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.337101 (+ 1021us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.337188 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:45.337210 (+ 22us) server_negotiation.cc:299] Negotiation successful
0806 14:58:45.337239 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
I20260806 14:58:45.337240 22007 tablet_service.cc:1467] Processing CreateTablet for tablet 61e212624536415faca745eb05ea9345 (DEFAULT_TABLE table=test-table [id=ac04ba35713b4338a97c422e925d8def]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:45.337566 22007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61e212624536415faca745eb05ea9345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:45.341387 22365 tablet_bootstrap.cc:492] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94: Bootstrap starting.
I20260806 14:58:45.342087 22365 tablet_bootstrap.cc:654] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:45.342145 22221 tablet_service.cc:1467] Processing CreateTablet for tablet 61e212624536415faca745eb05ea9345 (DEFAULT_TABLE table=test-table [id=ac04ba35713b4338a97c422e925d8def]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:45.342480 22365 log.cc:824] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:45.342567 22221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61e212624536415faca745eb05ea9345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:45.342675 22362 tablet_bootstrap.cc:492] T 61e212624536415faca745eb05ea9345 P 4c9b8da547274fca8cbca18abfebc2d0: Bootstrap starting.
I20260806 14:58:45.343379 22362 tablet_bootstrap.cc:654] T 61e212624536415faca745eb05ea9345 P 4c9b8da547274fca8cbca18abfebc2d0: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:45.343416 22365 tablet_bootstrap.cc:492] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94: No bootstrap required, opened a new log
I20260806 14:58:45.343478 22365 ts_tablet_manager.cc:1397] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 14:58:45.343776 22362 log.cc:824] T 61e212624536415faca745eb05ea9345 P 4c9b8da547274fca8cbca18abfebc2d0: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:45.344456 22362 tablet_bootstrap.cc:492] T 61e212624536415faca745eb05ea9345 P 4c9b8da547274fca8cbca18abfebc2d0: No bootstrap required, opened a new log
I20260806 14:58:45.344542 22362 ts_tablet_manager.cc:1397] T 61e212624536415faca745eb05ea9345 P 4c9b8da547274fca8cbca18abfebc2d0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 14:58:45.345618 22365 raft_consensus.cc:348] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } }
I20260806 14:58:45.345796 22365 raft_consensus.cc:374] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:45.345841 22365 raft_consensus.cc:729] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2751e00e4174c9ebcf59dd0ea26cb94, State: Initialized, Role: FOLLOWER
I20260806 14:58:45.345919 22362 raft_consensus.cc:348] T 61e212624536415faca745eb05ea9345 P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } }
I20260806 14:58:45.345985 22365 consensus_queue.cc:260] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } }
I20260806 14:58:45.346068 22362 raft_consensus.cc:374] T 61e212624536415faca745eb05ea9345 P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:45.346112 22362 raft_consensus.cc:729] T 61e212624536415faca745eb05ea9345 P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c9b8da547274fca8cbca18abfebc2d0, State: Initialized, Role: FOLLOWER
I20260806 14:58:45.346258 22365 ts_tablet_manager.cc:1428] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 14:58:45.346217 22362 consensus_queue.cc:260] T 61e212624536415faca745eb05ea9345 P 4c9b8da547274fca8cbca18abfebc2d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } }
I20260806 14:58:45.346390 22110 heartbeater.cc:503] Master 127.11.8.126:36493 was elected leader, sending a full tablet report...
I20260806 14:58:45.346534 22362 ts_tablet_manager.cc:1428] T 61e212624536415faca745eb05ea9345 P 4c9b8da547274fca8cbca18abfebc2d0: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 14:58:45.346592 21902 heartbeater.cc:503] Master 127.11.8.126:36493 was elected leader, sending a full tablet report...
I20260806 14:58:45.346779 22372 tablet_bootstrap.cc:492] T 61e212624536415faca745eb05ea9345 P 37ad3fe8e68b4d488ef0b44d31584401: Bootstrap starting.
I20260806 14:58:45.347558 22372 tablet_bootstrap.cc:654] T 61e212624536415faca745eb05ea9345 P 37ad3fe8e68b4d488ef0b44d31584401: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:45.347894 22372 log.cc:824] T 61e212624536415faca745eb05ea9345 P 37ad3fe8e68b4d488ef0b44d31584401: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:45.348670 22372 tablet_bootstrap.cc:492] T 61e212624536415faca745eb05ea9345 P 37ad3fe8e68b4d488ef0b44d31584401: No bootstrap required, opened a new log
I20260806 14:58:45.348774 22372 ts_tablet_manager.cc:1397] T 61e212624536415faca745eb05ea9345 P 37ad3fe8e68b4d488ef0b44d31584401: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 14:58:45.350296 22372 raft_consensus.cc:348] T 61e212624536415faca745eb05ea9345 P 37ad3fe8e68b4d488ef0b44d31584401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } }
I20260806 14:58:45.350483 22372 raft_consensus.cc:374] T 61e212624536415faca745eb05ea9345 P 37ad3fe8e68b4d488ef0b44d31584401 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:45.350539 22372 raft_consensus.cc:729] T 61e212624536415faca745eb05ea9345 P 37ad3fe8e68b4d488ef0b44d31584401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37ad3fe8e68b4d488ef0b44d31584401, State: Initialized, Role: FOLLOWER
I20260806 14:58:45.350682 22372 consensus_queue.cc:260] T 61e212624536415faca745eb05ea9345 P 37ad3fe8e68b4d488ef0b44d31584401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } }
I20260806 14:58:45.350961 22372 ts_tablet_manager.cc:1428] T 61e212624536415faca745eb05ea9345 P 37ad3fe8e68b4d488ef0b44d31584401: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 14:58:45.351042 22317 heartbeater.cc:503] Master 127.11.8.126:36493 was elected leader, sending a full tablet report...
W20260806 14:58:45.367806 22111 tablet.cc:2369] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 14:58:45.445070 21903 tablet.cc:2369] T 61e212624536415faca745eb05ea9345 P 4c9b8da547274fca8cbca18abfebc2d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 14:58:45.538008 22319 tablet.cc:2369] T 61e212624536415faca745eb05ea9345 P 37ad3fe8e68b4d488ef0b44d31584401: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 14:58:45.548763 22373 raft_consensus.cc:482] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 14:58:45.548884 22373 raft_consensus.cc:504] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } }
I20260806 14:58:45.549345 22373 leader_election.cc:288] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 37ad3fe8e68b4d488ef0b44d31584401 (127.11.8.67:40909), 4c9b8da547274fca8cbca18abfebc2d0 (127.11.8.65:34101)
I20260806 14:58:45.552520 22356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.549746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:34883 (local address 127.11.8.65:34101)
0806 14:58:45.549792 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:45.549794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:45.549809 (+ 15us) server_negotiation.cc:407] Connection header received
0806 14:58:45.549842 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:45.549844 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:45.549855 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:45.549960 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:45.550124 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.550847 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.551791 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.552154 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.552254 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.552265 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:45.552418 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 14:58:45.552452 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 14:58:45.552520 21948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.549627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:34101 (local address 127.11.8.66:34883)
0806 14:58:45.549700 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:45.549713 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:45.549768 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:45.549980 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:45.549981 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:45.549984 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:45.550055 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:45.550057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.550932 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.550933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:45.551686 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:45.551688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.552248 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.552249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:45.552343 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.552346 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 14:58:45.552413 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 14:58:45.552450 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 14:58:45.552876 22377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.549628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:40909 (local address 127.11.8.66:44499)
0806 14:58:45.549901 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:45.549909 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:45.549984 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:45.550180 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:45.550181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:45.550184 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:45.550276 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:45.550277 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.551107 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.551108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:45.551934 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:45.551936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.552546 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.552548 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:45.552629 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.552635 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 14:58:45.552699 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 14:58:45.552744 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":30,"thread_start_us":119,"threads_started":1}
I20260806 14:58:45.552963 21836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61e212624536415faca745eb05ea9345" candidate_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c9b8da547274fca8cbca18abfebc2d0" is_pre_election: true
I20260806 14:58:45.553088 22358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.549709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:44499 (local address 127.11.8.67:40909)
0806 14:58:45.549840 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:45.549842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:45.550008 (+ 166us) server_negotiation.cc:407] Connection header received
0806 14:58:45.550044 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:45.550067 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:45.550081 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:45.550157 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:45.550364 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.550991 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.552045 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.552436 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.552538 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.552550 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:45.552857 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 14:58:45.552908 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260806 14:58:45.553128 21836 raft_consensus.cc:2454] T 61e212624536415faca745eb05ea9345 P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2751e00e4174c9ebcf59dd0ea26cb94 in term 0.
I20260806 14:58:45.553385 22253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61e212624536415faca745eb05ea9345" candidate_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37ad3fe8e68b4d488ef0b44d31584401" is_pre_election: true
I20260806 14:58:45.553496 22253 raft_consensus.cc:2454] T 61e212624536415faca745eb05ea9345 P 37ad3fe8e68b4d488ef0b44d31584401 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2751e00e4174c9ebcf59dd0ea26cb94 in term 0.
I20260806 14:58:45.553772 21942 leader_election.cc:302] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c9b8da547274fca8cbca18abfebc2d0, e2751e00e4174c9ebcf59dd0ea26cb94; no voters:
I20260806 14:58:45.553937 22373 raft_consensus.cc:2781] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 14:58:45.554003 22373 raft_consensus.cc:482] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 14:58:45.554029 22373 raft_consensus.cc:3037] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:45.554903 22373 raft_consensus.cc:504] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } }
I20260806 14:58:45.555071 22373 leader_election.cc:288] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94 [CANDIDATE]: Term 1 election: Requested vote from peers 37ad3fe8e68b4d488ef0b44d31584401 (127.11.8.67:40909), 4c9b8da547274fca8cbca18abfebc2d0 (127.11.8.65:34101)
I20260806 14:58:45.555225 21836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61e212624536415faca745eb05ea9345" candidate_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c9b8da547274fca8cbca18abfebc2d0"
I20260806 14:58:45.555251 22253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61e212624536415faca745eb05ea9345" candidate_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37ad3fe8e68b4d488ef0b44d31584401"
I20260806 14:58:45.555311 21836 raft_consensus.cc:3037] T 61e212624536415faca745eb05ea9345 P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:45.555344 22253 raft_consensus.cc:3037] T 61e212624536415faca745eb05ea9345 P 37ad3fe8e68b4d488ef0b44d31584401 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:45.556140 22253 raft_consensus.cc:2454] T 61e212624536415faca745eb05ea9345 P 37ad3fe8e68b4d488ef0b44d31584401 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2751e00e4174c9ebcf59dd0ea26cb94 in term 1.
I20260806 14:58:45.556106 21836 raft_consensus.cc:2454] T 61e212624536415faca745eb05ea9345 P 4c9b8da547274fca8cbca18abfebc2d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2751e00e4174c9ebcf59dd0ea26cb94 in term 1.
I20260806 14:58:45.556392 21939 leader_election.cc:302] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37ad3fe8e68b4d488ef0b44d31584401, e2751e00e4174c9ebcf59dd0ea26cb94; no voters:
I20260806 14:58:45.556511 22373 raft_consensus.cc:2781] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:45.556686 22373 raft_consensus.cc:686] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94 [term 1 LEADER]: Becoming Leader. State: Replica: e2751e00e4174c9ebcf59dd0ea26cb94, State: Running, Role: LEADER
I20260806 14:58:45.556816 22373 consensus_queue.cc:237] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } }
I20260806 14:58:45.557626 21615 catalog_manager.cc:5495] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2751e00e4174c9ebcf59dd0ea26cb94 (127.11.8.66). New cstate: current_term: 1 leader_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } health_report { overall_health: UNKNOWN } } }
I20260806 14:58:45.617530 22361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.615266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58300 (local address 127.11.8.66:36205)
0806 14:58:45.615334 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:45.615336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:45.615353 (+ 17us) server_negotiation.cc:407] Connection header received
0806 14:58:45.615389 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:45.615390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:45.615410 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:45.615473 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 14:58:45.615606 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.616070 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.616777 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.616953 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.617002 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.617139 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 14:58:45.617236 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 14:58:45.617341 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:45.617355 (+ 14us) server_negotiation.cc:299] Negotiation successful
0806 14:58:45.617372 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 14:58:45.619640 22253 raft_consensus.cc:1261] T 61e212624536415faca745eb05ea9345 P 37ad3fe8e68b4d488ef0b44d31584401 [term 1 FOLLOWER]: Refusing update from remote peer e2751e00e4174c9ebcf59dd0ea26cb94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:45.619661 21836 raft_consensus.cc:1261] T 61e212624536415faca745eb05ea9345 P 4c9b8da547274fca8cbca18abfebc2d0 [term 1 FOLLOWER]: Refusing update from remote peer e2751e00e4174c9ebcf59dd0ea26cb94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:45.619956 22373 consensus_queue.cc:1035] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:45.620126 22380 consensus_queue.cc:1035] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:45.623344 22385 mvcc.cc:205] Tried to move back new op lower bound from 7315572021734666240 to 7315572021481238528. Current Snapshot: MvccSnapshot[applied={T|T < 7315572021734666240}]
I20260806 14:58:45.624562 21615 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:58:45.624665 21615 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:58:45.625429 21615 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b79710c047b8421f88b764f7af7025c6: Sending DeleteTablet for 3 replicas of tablet 61e212624536415faca745eb05ea9345
I20260806 14:58:45.625780 22221 tablet_service.cc:1514] Processing DeleteTablet for tablet 61e212624536415faca745eb05ea9345 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:45 UTC) from {username='oryx', principal='oryx/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:42396
I20260806 14:58:45.625792 21810 tablet_service.cc:1514] Processing DeleteTablet for tablet 61e212624536415faca745eb05ea9345 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:45 UTC) from {username='oryx', principal='oryx/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:39080
I20260806 14:58:45.625984 21615 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38302:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 14:58:45.626036 22007 tablet_service.cc:1514] Processing DeleteTablet for tablet 61e212624536415faca745eb05ea9345 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:45 UTC) from {username='oryx', principal='oryx/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:58284
W20260806 14:58:45.626137 21615 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 14:58:45.626271 22400 tablet_replica.cc:330] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94: stopping tablet replica
I20260806 14:58:45.626336 22397 tablet_replica.cc:330] T 61e212624536415faca745eb05ea9345 P 4c9b8da547274fca8cbca18abfebc2d0: stopping tablet replica
I20260806 14:58:45.626432 22397 raft_consensus.cc:2229] T 61e212624536415faca745eb05ea9345 P 4c9b8da547274fca8cbca18abfebc2d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:58:45.626542 22397 raft_consensus.cc:2258] T 61e212624536415faca745eb05ea9345 P 4c9b8da547274fca8cbca18abfebc2d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:45.626607 22398 tablet_replica.cc:330] T 61e212624536415faca745eb05ea9345 P 37ad3fe8e68b4d488ef0b44d31584401: stopping tablet replica
I20260806 14:58:45.626752 22398 raft_consensus.cc:2229] T 61e212624536415faca745eb05ea9345 P 37ad3fe8e68b4d488ef0b44d31584401 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:58:45.627136 22398 raft_consensus.cc:2258] T 61e212624536415faca745eb05ea9345 P 37ad3fe8e68b4d488ef0b44d31584401 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:45.626370 22400 raft_consensus.cc:2229] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94 [term 1 LEADER]: Raft consensus shutting down.
I20260806 14:58:45.627358 22400 raft_consensus.cc:2258] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:45.627550 21810 tablet_service.cc:1467] Processing CreateTablet for tablet 18216016191f44ab83f39c31f729e6fc (DEFAULT_TABLE table=test-table [id=b2a79582d40c44969762190bc27b7d6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:45.627565 22221 tablet_service.cc:1467] Processing CreateTablet for tablet 18216016191f44ab83f39c31f729e6fc (DEFAULT_TABLE table=test-table [id=b2a79582d40c44969762190bc27b7d6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:45.627687 22221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18216016191f44ab83f39c31f729e6fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:45.627677 21810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18216016191f44ab83f39c31f729e6fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:45.627815 22400 ts_tablet_manager.cc:1905] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:45.629119 22362 tablet_bootstrap.cc:492] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0: Bootstrap starting.
I20260806 14:58:45.629177 22007 tablet_service.cc:1467] Processing CreateTablet for tablet 18216016191f44ab83f39c31f729e6fc (DEFAULT_TABLE table=test-table [id=b2a79582d40c44969762190bc27b7d6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:45.629274 22007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18216016191f44ab83f39c31f729e6fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:45.629415 22372 tablet_bootstrap.cc:492] T 18216016191f44ab83f39c31f729e6fc P 37ad3fe8e68b4d488ef0b44d31584401: Bootstrap starting.
I20260806 14:58:45.629539 22362 tablet_bootstrap.cc:654] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:45.629859 22372 tablet_bootstrap.cc:654] T 18216016191f44ab83f39c31f729e6fc P 37ad3fe8e68b4d488ef0b44d31584401: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:45.630260 22362 tablet_bootstrap.cc:492] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0: No bootstrap required, opened a new log
I20260806 14:58:45.630317 22362 ts_tablet_manager.cc:1397] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 14:58:45.630496 22362 raft_consensus.cc:348] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:45.630584 22362 raft_consensus.cc:374] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:45.630613 22362 raft_consensus.cc:729] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c9b8da547274fca8cbca18abfebc2d0, State: Initialized, Role: FOLLOWER
I20260806 14:58:45.630640 22372 tablet_bootstrap.cc:492] T 18216016191f44ab83f39c31f729e6fc P 37ad3fe8e68b4d488ef0b44d31584401: No bootstrap required, opened a new log
I20260806 14:58:45.630656 22362 consensus_queue.cc:260] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:45.630707 22372 ts_tablet_manager.cc:1397] T 18216016191f44ab83f39c31f729e6fc P 37ad3fe8e68b4d488ef0b44d31584401: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:45.630769 22362 ts_tablet_manager.cc:1428] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 14:58:45.630877 22372 raft_consensus.cc:348] T 18216016191f44ab83f39c31f729e6fc P 37ad3fe8e68b4d488ef0b44d31584401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:45.630935 22398 ts_tablet_manager.cc:1905] T 61e212624536415faca745eb05ea9345 P 37ad3fe8e68b4d488ef0b44d31584401: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:45.630959 22372 raft_consensus.cc:374] T 18216016191f44ab83f39c31f729e6fc P 37ad3fe8e68b4d488ef0b44d31584401 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:45.630980 22372 raft_consensus.cc:729] T 18216016191f44ab83f39c31f729e6fc P 37ad3fe8e68b4d488ef0b44d31584401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37ad3fe8e68b4d488ef0b44d31584401, State: Initialized, Role: FOLLOWER
I20260806 14:58:45.631023 22372 consensus_queue.cc:260] T 18216016191f44ab83f39c31f729e6fc P 37ad3fe8e68b4d488ef0b44d31584401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:45.631124 22372 ts_tablet_manager.cc:1428] T 18216016191f44ab83f39c31f729e6fc P 37ad3fe8e68b4d488ef0b44d31584401: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:45.631242 22365 tablet_bootstrap.cc:492] T 18216016191f44ab83f39c31f729e6fc P e2751e00e4174c9ebcf59dd0ea26cb94: Bootstrap starting.
I20260806 14:58:45.631629 22365 tablet_bootstrap.cc:654] T 18216016191f44ab83f39c31f729e6fc P e2751e00e4174c9ebcf59dd0ea26cb94: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:45.632477 22398 ts_tablet_manager.cc:1918] T 61e212624536415faca745eb05ea9345 P 37ad3fe8e68b4d488ef0b44d31584401: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 14:58:45.632553 22398 log.cc:1196] T 61e212624536415faca745eb05ea9345 P 37ad3fe8e68b4d488ef0b44d31584401: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-2/wal/wals/61e212624536415faca745eb05ea9345
I20260806 14:58:45.632835 22365 tablet_bootstrap.cc:492] T 18216016191f44ab83f39c31f729e6fc P e2751e00e4174c9ebcf59dd0ea26cb94: No bootstrap required, opened a new log
I20260806 14:58:45.632833 22398 ts_tablet_manager.cc:1939] T 61e212624536415faca745eb05ea9345 P 37ad3fe8e68b4d488ef0b44d31584401: Deleting consensus metadata
I20260806 14:58:45.632886 22365 ts_tablet_manager.cc:1397] T 18216016191f44ab83f39c31f729e6fc P e2751e00e4174c9ebcf59dd0ea26cb94: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 14:58:45.633029 22365 raft_consensus.cc:348] T 18216016191f44ab83f39c31f729e6fc P e2751e00e4174c9ebcf59dd0ea26cb94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:45.633102 22365 raft_consensus.cc:374] T 18216016191f44ab83f39c31f729e6fc P e2751e00e4174c9ebcf59dd0ea26cb94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:45.633121 22365 raft_consensus.cc:729] T 18216016191f44ab83f39c31f729e6fc P e2751e00e4174c9ebcf59dd0ea26cb94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2751e00e4174c9ebcf59dd0ea26cb94, State: Initialized, Role: FOLLOWER
I20260806 14:58:45.633186 22365 consensus_queue.cc:260] T 18216016191f44ab83f39c31f729e6fc P e2751e00e4174c9ebcf59dd0ea26cb94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:45.633270 22400 ts_tablet_manager.cc:1918] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 14:58:45.633293 22365 ts_tablet_manager.cc:1428] T 18216016191f44ab83f39c31f729e6fc P e2751e00e4174c9ebcf59dd0ea26cb94: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:45.633334 22400 log.cc:1196] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-1/wal/wals/61e212624536415faca745eb05ea9345
I20260806 14:58:45.633464 21589 catalog_manager.cc:4841] TS 37ad3fe8e68b4d488ef0b44d31584401 (127.11.8.67:40909): tablet 61e212624536415faca745eb05ea9345 (table test-table [id=ac04ba35713b4338a97c422e925d8def]) successfully deleted
I20260806 14:58:45.633610 22400 ts_tablet_manager.cc:1939] T 61e212624536415faca745eb05ea9345 P e2751e00e4174c9ebcf59dd0ea26cb94: Deleting consensus metadata
I20260806 14:58:45.633976 22397 ts_tablet_manager.cc:1905] T 61e212624536415faca745eb05ea9345 P 4c9b8da547274fca8cbca18abfebc2d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:45.633980 21589 catalog_manager.cc:4841] TS e2751e00e4174c9ebcf59dd0ea26cb94 (127.11.8.66:36205): tablet 61e212624536415faca745eb05ea9345 (table test-table [id=ac04ba35713b4338a97c422e925d8def]) successfully deleted
I20260806 14:58:45.635551 22397 ts_tablet_manager.cc:1918] T 61e212624536415faca745eb05ea9345 P 4c9b8da547274fca8cbca18abfebc2d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 14:58:45.635627 22397 log.cc:1196] T 61e212624536415faca745eb05ea9345 P 4c9b8da547274fca8cbca18abfebc2d0: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-0/wal/wals/61e212624536415faca745eb05ea9345
I20260806 14:58:45.635861 22397 ts_tablet_manager.cc:1939] T 61e212624536415faca745eb05ea9345 P 4c9b8da547274fca8cbca18abfebc2d0: Deleting consensus metadata
I20260806 14:58:45.636215 21594 catalog_manager.cc:4841] TS 4c9b8da547274fca8cbca18abfebc2d0 (127.11.8.65:34101): tablet 61e212624536415faca745eb05ea9345 (table test-table [id=ac04ba35713b4338a97c422e925d8def]) successfully deleted
I20260806 14:58:45.653530 22374 raft_consensus.cc:482] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 14:58:45.653630 22374 raft_consensus.cc:504] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:45.653972 22374 leader_election.cc:288] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2751e00e4174c9ebcf59dd0ea26cb94 (127.11.8.66:36205), 37ad3fe8e68b4d488ef0b44d31584401 (127.11.8.67:40909)
I20260806 14:58:45.658082 22361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.654254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:55657 (local address 127.11.8.66:36205)
0806 14:58:45.654303 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:45.654306 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:45.654371 (+ 65us) server_negotiation.cc:407] Connection header received
0806 14:58:45.654576 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:45.654577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:45.654588 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:45.654622 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:45.654850 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.655618 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.657120 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.657675 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.657723 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.657733 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:45.657972 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 14:58:45.658007 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 14:58:45.658332 22407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.654174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:36205 (local address 127.11.8.65:55657)
0806 14:58:45.654313 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:45.654325 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:45.654387 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:45.654702 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:45.654702 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:45.654703 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:45.654771 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:45.654773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.655718 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.655719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:45.657016 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:45.657019 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.657807 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.657808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:45.657890 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.657894 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 14:58:45.658198 (+ 304us) client_negotiation.cc:240] Negotiation successful
0806 14:58:45.658241 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":94,"threads_started":1}
I20260806 14:58:45.658735 22038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18216016191f44ab83f39c31f729e6fc" candidate_uuid: "4c9b8da547274fca8cbca18abfebc2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" is_pre_election: true
I20260806 14:58:45.658960 22038 raft_consensus.cc:2454] T 18216016191f44ab83f39c31f729e6fc P e2751e00e4174c9ebcf59dd0ea26cb94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c9b8da547274fca8cbca18abfebc2d0 in term 0.
I20260806 14:58:45.659194 21738 leader_election.cc:302] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c9b8da547274fca8cbca18abfebc2d0, e2751e00e4174c9ebcf59dd0ea26cb94; no voters:
I20260806 14:58:45.659317 22358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.654493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:60747 (local address 127.11.8.67:40909)
0806 14:58:45.654546 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:45.654548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:45.654582 (+ 34us) server_negotiation.cc:407] Connection header received
0806 14:58:45.654609 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:45.654610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:45.654620 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:45.654664 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:58:45.655715 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.656394 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.657653 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.658237 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.658287 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.658296 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:45.659215 (+ 919us) server_negotiation.cc:299] Negotiation successful
0806 14:58:45.659256 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 14:58:45.659323 22374 raft_consensus.cc:2781] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 14:58:45.659377 22374 raft_consensus.cc:482] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 14:58:45.659518 22374 raft_consensus.cc:3037] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:45.660396 22374 raft_consensus.cc:504] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:45.660543 22374 leader_election.cc:288] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0 [CANDIDATE]: Term 1 election: Requested vote from peers e2751e00e4174c9ebcf59dd0ea26cb94 (127.11.8.66:36205), 37ad3fe8e68b4d488ef0b44d31584401 (127.11.8.67:40909)
I20260806 14:58:45.659335 22409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.654382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:40909 (local address 127.11.8.65:60747)
0806 14:58:45.654482 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:45.654492 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:45.654542 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:45.654680 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:45.654681 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:45.654683 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:58:45.654771 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:45.654774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.656496 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.656498 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:45.657534 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:45.657537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.659045 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.659048 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:45.659123 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.659127 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 14:58:45.659205 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 14:58:45.659245 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":4,"thread_start_us":54,"threads_started":1}
I20260806 14:58:45.660766 22038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18216016191f44ab83f39c31f729e6fc" candidate_uuid: "4c9b8da547274fca8cbca18abfebc2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94"
I20260806 14:58:45.660845 22038 raft_consensus.cc:3037] T 18216016191f44ab83f39c31f729e6fc P e2751e00e4174c9ebcf59dd0ea26cb94 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:45.660818 22253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18216016191f44ab83f39c31f729e6fc" candidate_uuid: "4c9b8da547274fca8cbca18abfebc2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37ad3fe8e68b4d488ef0b44d31584401" is_pre_election: true
I20260806 14:58:45.660892 22253 raft_consensus.cc:2454] T 18216016191f44ab83f39c31f729e6fc P 37ad3fe8e68b4d488ef0b44d31584401 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c9b8da547274fca8cbca18abfebc2d0 in term 0.
I20260806 14:58:45.660853 22252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18216016191f44ab83f39c31f729e6fc" candidate_uuid: "4c9b8da547274fca8cbca18abfebc2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37ad3fe8e68b4d488ef0b44d31584401"
I20260806 14:58:45.660943 22252 raft_consensus.cc:3037] T 18216016191f44ab83f39c31f729e6fc P 37ad3fe8e68b4d488ef0b44d31584401 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:45.661531 22038 raft_consensus.cc:2454] T 18216016191f44ab83f39c31f729e6fc P e2751e00e4174c9ebcf59dd0ea26cb94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c9b8da547274fca8cbca18abfebc2d0 in term 1.
I20260806 14:58:45.661684 21738 leader_election.cc:302] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c9b8da547274fca8cbca18abfebc2d0, e2751e00e4174c9ebcf59dd0ea26cb94; no voters:
I20260806 14:58:45.661762 22252 raft_consensus.cc:2454] T 18216016191f44ab83f39c31f729e6fc P 37ad3fe8e68b4d488ef0b44d31584401 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c9b8da547274fca8cbca18abfebc2d0 in term 1.
I20260806 14:58:45.661774 22374 raft_consensus.cc:2781] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:45.661908 22374 raft_consensus.cc:686] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0 [term 1 LEADER]: Becoming Leader. State: Replica: 4c9b8da547274fca8cbca18abfebc2d0, State: Running, Role: LEADER
I20260806 14:58:45.661986 22374 consensus_queue.cc:237] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:45.662719 21615 catalog_manager.cc:5495] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c9b8da547274fca8cbca18abfebc2d0 (127.11.8.65). New cstate: current_term: 1 leader_uuid: "4c9b8da547274fca8cbca18abfebc2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } health_report { overall_health: UNKNOWN } } }
Aug 06 14:58:45 dist-test-slave-68xk krb5kdc[21554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028324, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.126@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM
I20260806 14:58:45.691699 21685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.683343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38316 (local address 127.11.8.126:36493)
0806 14:58:45.683427 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:45.683430 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:45.683581 (+ 151us) server_negotiation.cc:407] Connection header received
0806 14:58:45.683649 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:45.683650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:45.683659 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:45.683711 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:45.684280 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.684765 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.685825 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.686006 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.686065 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.688510 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:45.688512 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:45.688513 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:45.688521 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:45.690335 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:45.690843 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:45.690844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:45.690845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:45.690946 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:45.691203 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:45.691204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:45.691204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:45.691314 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:45.691371 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:45.691552 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 14:58:45.691634 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 14:58:45.691813 22418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.683275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:36493 (local address 127.0.0.1:38316)
0806 14:58:45.683493 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:45.683506 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:45.683583 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:45.683743 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:45.683745 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:45.684137 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:45.684197 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:45.684200 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.684852 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.684853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:45.685723 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:45.685725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.686085 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.686087 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:45.686140 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.686764 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:45.686770 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:45.688384 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:45.690440 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:45.690448 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:45.690450 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:45.690710 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:45.691050 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:45.691051 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:45.691051 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:45.691129 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:45.691388 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:45.691396 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:45.691450 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 14:58:45.691534 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 14:58:45.691711 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":56,"thread_start_us":130,"threads_started":1}
I20260806 14:58:45.694228 21615 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:38316:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 14:58:45.694557 21615 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 14:58:45.696254 22007 tablet_service.cc:1467] Processing CreateTablet for tablet 9d096398b8ec4296898c4f1935ed64cb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b0ce811ff6e0470796e833b8642d94d3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 14:58:45.696267 21810 tablet_service.cc:1467] Processing CreateTablet for tablet 9d096398b8ec4296898c4f1935ed64cb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b0ce811ff6e0470796e833b8642d94d3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 14:58:45.696264 22221 tablet_service.cc:1467] Processing CreateTablet for tablet 9d096398b8ec4296898c4f1935ed64cb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b0ce811ff6e0470796e833b8642d94d3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 14:58:45.696434 22221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d096398b8ec4296898c4f1935ed64cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:45.696436 22007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d096398b8ec4296898c4f1935ed64cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:45.696435 21810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d096398b8ec4296898c4f1935ed64cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:45.698047 22362 tablet_bootstrap.cc:492] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0: Bootstrap starting.
I20260806 14:58:45.698057 22372 tablet_bootstrap.cc:492] T 9d096398b8ec4296898c4f1935ed64cb P 37ad3fe8e68b4d488ef0b44d31584401: Bootstrap starting.
I20260806 14:58:45.698566 22362 tablet_bootstrap.cc:654] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:45.698566 22372 tablet_bootstrap.cc:654] T 9d096398b8ec4296898c4f1935ed64cb P 37ad3fe8e68b4d488ef0b44d31584401: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:45.698985 22365 tablet_bootstrap.cc:492] T 9d096398b8ec4296898c4f1935ed64cb P e2751e00e4174c9ebcf59dd0ea26cb94: Bootstrap starting.
I20260806 14:58:45.699375 22362 tablet_bootstrap.cc:492] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0: No bootstrap required, opened a new log
I20260806 14:58:45.699424 22362 ts_tablet_manager.cc:1397] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 14:58:45.699467 22365 tablet_bootstrap.cc:654] T 9d096398b8ec4296898c4f1935ed64cb P e2751e00e4174c9ebcf59dd0ea26cb94: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:45.699615 22362 raft_consensus.cc:348] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:45.699707 22362 raft_consensus.cc:374] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:45.699731 22362 raft_consensus.cc:729] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c9b8da547274fca8cbca18abfebc2d0, State: Initialized, Role: FOLLOWER
I20260806 14:58:45.699787 22362 consensus_queue.cc:260] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:45.699905 22362 ts_tablet_manager.cc:1428] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 14:58:45.700064 22374 tablet_replica.cc:439] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } } }
I20260806 14:58:45.700143 22374 tablet_replica.cc:442] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 14:58:45.700155 22365 tablet_bootstrap.cc:492] T 9d096398b8ec4296898c4f1935ed64cb P e2751e00e4174c9ebcf59dd0ea26cb94: No bootstrap required, opened a new log
I20260806 14:58:45.700202 22365 ts_tablet_manager.cc:1397] T 9d096398b8ec4296898c4f1935ed64cb P e2751e00e4174c9ebcf59dd0ea26cb94: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 14:58:45.700340 22365 raft_consensus.cc:348] T 9d096398b8ec4296898c4f1935ed64cb P e2751e00e4174c9ebcf59dd0ea26cb94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:45.700420 22365 raft_consensus.cc:374] T 9d096398b8ec4296898c4f1935ed64cb P e2751e00e4174c9ebcf59dd0ea26cb94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:45.700449 22365 raft_consensus.cc:729] T 9d096398b8ec4296898c4f1935ed64cb P e2751e00e4174c9ebcf59dd0ea26cb94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2751e00e4174c9ebcf59dd0ea26cb94, State: Initialized, Role: FOLLOWER
I20260806 14:58:45.700490 22365 consensus_queue.cc:260] T 9d096398b8ec4296898c4f1935ed64cb P e2751e00e4174c9ebcf59dd0ea26cb94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:45.700579 22365 ts_tablet_manager.cc:1428] T 9d096398b8ec4296898c4f1935ed64cb P e2751e00e4174c9ebcf59dd0ea26cb94: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:45.700651 22393 tablet_replica.cc:439] T 9d096398b8ec4296898c4f1935ed64cb P e2751e00e4174c9ebcf59dd0ea26cb94: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } } }
I20260806 14:58:45.700748 22393 tablet_replica.cc:442] T 9d096398b8ec4296898c4f1935ed64cb P e2751e00e4174c9ebcf59dd0ea26cb94: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 14:58:45.701359 22372 tablet_bootstrap.cc:492] T 9d096398b8ec4296898c4f1935ed64cb P 37ad3fe8e68b4d488ef0b44d31584401: No bootstrap required, opened a new log
I20260806 14:58:45.701418 22372 ts_tablet_manager.cc:1397] T 9d096398b8ec4296898c4f1935ed64cb P 37ad3fe8e68b4d488ef0b44d31584401: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260806 14:58:45.701560 22372 raft_consensus.cc:348] T 9d096398b8ec4296898c4f1935ed64cb P 37ad3fe8e68b4d488ef0b44d31584401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:45.701630 22372 raft_consensus.cc:374] T 9d096398b8ec4296898c4f1935ed64cb P 37ad3fe8e68b4d488ef0b44d31584401 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:45.701651 22372 raft_consensus.cc:729] T 9d096398b8ec4296898c4f1935ed64cb P 37ad3fe8e68b4d488ef0b44d31584401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37ad3fe8e68b4d488ef0b44d31584401, State: Initialized, Role: FOLLOWER
I20260806 14:58:45.701692 22372 consensus_queue.cc:260] T 9d096398b8ec4296898c4f1935ed64cb P 37ad3fe8e68b4d488ef0b44d31584401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:45.701795 22372 ts_tablet_manager.cc:1428] T 9d096398b8ec4296898c4f1935ed64cb P 37ad3fe8e68b4d488ef0b44d31584401: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:45.701874 22376 tablet_replica.cc:439] T 9d096398b8ec4296898c4f1935ed64cb P 37ad3fe8e68b4d488ef0b44d31584401: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } } }
I20260806 14:58:45.701978 22376 tablet_replica.cc:442] T 9d096398b8ec4296898c4f1935ed64cb P 37ad3fe8e68b4d488ef0b44d31584401: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 14:58:45.922143 22374 raft_consensus.cc:482] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 14:58:45.922257 22374 raft_consensus.cc:504] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:45.922516 22374 leader_election.cc:288] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2751e00e4174c9ebcf59dd0ea26cb94 (127.11.8.66:36205), 37ad3fe8e68b4d488ef0b44d31584401 (127.11.8.67:40909)
I20260806 14:58:45.922806 22038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d096398b8ec4296898c4f1935ed64cb" candidate_uuid: "4c9b8da547274fca8cbca18abfebc2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" is_pre_election: true
I20260806 14:58:45.922912 22038 raft_consensus.cc:2454] T 9d096398b8ec4296898c4f1935ed64cb P e2751e00e4174c9ebcf59dd0ea26cb94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c9b8da547274fca8cbca18abfebc2d0 in term 0.
I20260806 14:58:45.922842 22252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d096398b8ec4296898c4f1935ed64cb" candidate_uuid: "4c9b8da547274fca8cbca18abfebc2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37ad3fe8e68b4d488ef0b44d31584401" is_pre_election: true
I20260806 14:58:45.922971 22252 raft_consensus.cc:2454] T 9d096398b8ec4296898c4f1935ed64cb P 37ad3fe8e68b4d488ef0b44d31584401 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c9b8da547274fca8cbca18abfebc2d0 in term 0.
I20260806 14:58:45.923138 21738 leader_election.cc:302] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c9b8da547274fca8cbca18abfebc2d0, e2751e00e4174c9ebcf59dd0ea26cb94; no voters:
I20260806 14:58:45.923300 22374 raft_consensus.cc:2781] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 14:58:45.923364 22374 raft_consensus.cc:482] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 14:58:45.923398 22374 raft_consensus.cc:3037] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:45.924091 22374 raft_consensus.cc:504] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:45.924226 22374 leader_election.cc:288] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0 [CANDIDATE]: Term 1 election: Requested vote from peers e2751e00e4174c9ebcf59dd0ea26cb94 (127.11.8.66:36205), 37ad3fe8e68b4d488ef0b44d31584401 (127.11.8.67:40909)
I20260806 14:58:45.924440 22038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d096398b8ec4296898c4f1935ed64cb" candidate_uuid: "4c9b8da547274fca8cbca18abfebc2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94"
I20260806 14:58:45.924443 22252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d096398b8ec4296898c4f1935ed64cb" candidate_uuid: "4c9b8da547274fca8cbca18abfebc2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37ad3fe8e68b4d488ef0b44d31584401"
I20260806 14:58:45.924539 22038 raft_consensus.cc:3037] T 9d096398b8ec4296898c4f1935ed64cb P e2751e00e4174c9ebcf59dd0ea26cb94 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:45.924540 22252 raft_consensus.cc:3037] T 9d096398b8ec4296898c4f1935ed64cb P 37ad3fe8e68b4d488ef0b44d31584401 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:45.925418 22038 raft_consensus.cc:2454] T 9d096398b8ec4296898c4f1935ed64cb P e2751e00e4174c9ebcf59dd0ea26cb94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c9b8da547274fca8cbca18abfebc2d0 in term 1.
I20260806 14:58:45.925429 22252 raft_consensus.cc:2454] T 9d096398b8ec4296898c4f1935ed64cb P 37ad3fe8e68b4d488ef0b44d31584401 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c9b8da547274fca8cbca18abfebc2d0 in term 1.
I20260806 14:58:45.925627 21738 leader_election.cc:302] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c9b8da547274fca8cbca18abfebc2d0, e2751e00e4174c9ebcf59dd0ea26cb94; no voters:
I20260806 14:58:45.925753 22374 raft_consensus.cc:2781] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:45.925819 22374 raft_consensus.cc:686] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0 [term 1 LEADER]: Becoming Leader. State: Replica: 4c9b8da547274fca8cbca18abfebc2d0, State: Running, Role: LEADER
I20260806 14:58:45.925876 22374 consensus_queue.cc:237] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:45.926240 22411 tablet_replica.cc:439] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0: TxnStatusTablet state changed. Reason: New leader 4c9b8da547274fca8cbca18abfebc2d0. Latest consensus state: current_term: 1 leader_uuid: "4c9b8da547274fca8cbca18abfebc2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } } }
I20260806 14:58:45.926343 22411 tablet_replica.cc:442] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0: This TxnStatusTablet replica's current role is: LEADER
I20260806 14:58:45.926558 22426 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 14:58:45.926681 22426 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 14:58:45.926615 21615 catalog_manager.cc:5495] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c9b8da547274fca8cbca18abfebc2d0 (127.11.8.65). New cstate: current_term: 1 leader_uuid: "4c9b8da547274fca8cbca18abfebc2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } health_report { overall_health: UNKNOWN } } }
I20260806 14:58:45.988204 22418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.985625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:34101 (local address 127.0.0.1:39092)
0806 14:58:45.985701 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:45.985713 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:45.985768 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:45.985961 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:45.985962 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:45.985964 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 14:58:45.986016 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:45.986018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.986668 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.986669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:45.987250 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:45.987252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.987612 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.987613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:45.987678 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.987693 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 14:58:45.988022 (+ 329us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 14:58:45.988032 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 14:58:45.988074 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 14:58:45.988122 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 14:58:45.988205 22356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.985699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39092 (local address 127.11.8.65:34101)
0806 14:58:45.985759 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:45.985761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:45.985772 (+ 11us) server_negotiation.cc:407] Connection header received
0806 14:58:45.985858 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:45.985859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:45.985883 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:45.985953 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 14:58:45.986074 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.986581 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.987344 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.987537 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.987612 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.987802 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 14:58:45.987949 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 14:58:45.988025 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:45.988090 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 14:58:45.988115 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 14:58:45.989441 22252 raft_consensus.cc:1261] T 9d096398b8ec4296898c4f1935ed64cb P 37ad3fe8e68b4d488ef0b44d31584401 [term 1 FOLLOWER]: Refusing update from remote peer 4c9b8da547274fca8cbca18abfebc2d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:45.989490 22038 raft_consensus.cc:1261] T 9d096398b8ec4296898c4f1935ed64cb P e2751e00e4174c9ebcf59dd0ea26cb94 [term 1 FOLLOWER]: Refusing update from remote peer 4c9b8da547274fca8cbca18abfebc2d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:45.989910 22411 consensus_queue.cc:1035] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:45.990049 22374 consensus_queue.cc:1035] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:45.990808 22393 tablet_replica.cc:439] T 9d096398b8ec4296898c4f1935ed64cb P e2751e00e4174c9ebcf59dd0ea26cb94: TxnStatusTablet state changed. Reason: New leader 4c9b8da547274fca8cbca18abfebc2d0. Latest consensus state: current_term: 1 leader_uuid: "4c9b8da547274fca8cbca18abfebc2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } } }
I20260806 14:58:45.990913 22393 tablet_replica.cc:442] T 9d096398b8ec4296898c4f1935ed64cb P e2751e00e4174c9ebcf59dd0ea26cb94: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 14:58:45.990854 22376 tablet_replica.cc:439] T 9d096398b8ec4296898c4f1935ed64cb P 37ad3fe8e68b4d488ef0b44d31584401: TxnStatusTablet state changed. Reason: New leader 4c9b8da547274fca8cbca18abfebc2d0. Latest consensus state: current_term: 1 leader_uuid: "4c9b8da547274fca8cbca18abfebc2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } } }
I20260806 14:58:45.990940 22376 tablet_replica.cc:442] T 9d096398b8ec4296898c4f1935ed64cb P 37ad3fe8e68b4d488ef0b44d31584401: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 14:58:45.991755 22374 tablet_replica.cc:439] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4c9b8da547274fca8cbca18abfebc2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } } }
I20260806 14:58:45.991864 22411 tablet_replica.cc:439] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4c9b8da547274fca8cbca18abfebc2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } } }
I20260806 14:58:45.991962 22411 tablet_replica.cc:442] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0: This TxnStatusTablet replica's current role is: LEADER
I20260806 14:58:45.991852 22374 tablet_replica.cc:442] T 9d096398b8ec4296898c4f1935ed64cb P 4c9b8da547274fca8cbca18abfebc2d0: This TxnStatusTablet replica's current role is: LEADER
I20260806 14:58:45.992269 22384 mvcc.cc:205] Tried to move back new op lower bound from 7315572023250464768 to 7315572022992719872. Current Snapshot: MvccSnapshot[applied={T|T < 7315572023250464768}]
I20260806 14:58:45.992369 22393 tablet_replica.cc:439] T 9d096398b8ec4296898c4f1935ed64cb P e2751e00e4174c9ebcf59dd0ea26cb94: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4c9b8da547274fca8cbca18abfebc2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } } }
I20260806 14:58:45.992446 22393 tablet_replica.cc:442] T 9d096398b8ec4296898c4f1935ed64cb P e2751e00e4174c9ebcf59dd0ea26cb94: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 14:58:45.992736 22386 mvcc.cc:205] Tried to move back new op lower bound from 7315572023250464768 to 7315572022992719872. Current Snapshot: MvccSnapshot[applied={T|T < 7315572023250464768}]
I20260806 14:58:45.992823 22376 tablet_replica.cc:439] T 9d096398b8ec4296898c4f1935ed64cb P 37ad3fe8e68b4d488ef0b44d31584401: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4c9b8da547274fca8cbca18abfebc2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } } }
I20260806 14:58:45.992911 22376 tablet_replica.cc:442] T 9d096398b8ec4296898c4f1935ed64cb P 37ad3fe8e68b4d488ef0b44d31584401: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 14:58:45.995884 22356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.993704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39104 (local address 127.11.8.65:34101)
0806 14:58:45.993745 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:45.993747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:45.993753 (+ 6us) server_negotiation.cc:407] Connection header received
0806 14:58:45.993771 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:45.993772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:45.993782 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:45.993854 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 14:58:45.993994 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.994496 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.995205 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.995410 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.995487 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:45.995647 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 14:58:45.995679 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 14:58:45.995727 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:45.995793 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 14:58:45.995812 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 14:58:46.001322 22356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.998637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:43181 (local address 127.11.8.65:34101)
0806 14:58:45.998695 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:45.998697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:45.998819 (+ 122us) server_negotiation.cc:407] Connection header received
0806 14:58:45.998844 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:45.998845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:45.998856 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:45.998919 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 14:58:45.999004 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.999737 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:46.000490 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:46.000803 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:46.000890 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:46.001046 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 14:58:46.001081 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 14:58:46.001147 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:46.001193 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 14:58:46.001214 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 14:58:46.001338 22439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:45.998567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:34101 (local address 127.11.8.65:43181)
0806 14:58:45.998695 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:45.998705 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:45.998777 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:45.998920 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:45.998920 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:45.998922 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 14:58:45.998962 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:45.998963 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:45.999820 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:45.999820 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:46.000396 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:46.000398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:46.000901 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:46.000902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:46.000949 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:46.000962 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 14:58:46.001151 (+ 189us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 14:58:46.001153 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 14:58:46.001186 (+ 33us) client_negotiation.cc:240] Negotiation successful
0806 14:58:46.001217 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":69,"threads_started":1}
I20260806 14:58:46.003383 22038 raft_consensus.cc:1261] T 18216016191f44ab83f39c31f729e6fc P e2751e00e4174c9ebcf59dd0ea26cb94 [term 1 FOLLOWER]: Refusing update from remote peer 4c9b8da547274fca8cbca18abfebc2d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:46.003480 22252 raft_consensus.cc:1261] T 18216016191f44ab83f39c31f729e6fc P 37ad3fe8e68b4d488ef0b44d31584401 [term 1 FOLLOWER]: Refusing update from remote peer 4c9b8da547274fca8cbca18abfebc2d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:46.003713 22374 consensus_queue.cc:1035] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:46.003788 22411 consensus_queue.cc:1035] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:46.016592 21615 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:58:46.016710 21615 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:58:46.017345 21615 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b79710c047b8421f88b764f7af7025c6: Sending DeleteTablet for 3 replicas of tablet 18216016191f44ab83f39c31f729e6fc
I20260806 14:58:46.017663 22007 tablet_service.cc:1514] Processing DeleteTablet for tablet 18216016191f44ab83f39c31f729e6fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:46 UTC) from {username='oryx', principal='oryx/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:58284
I20260806 14:58:46.017674 21810 tablet_service.cc:1514] Processing DeleteTablet for tablet 18216016191f44ab83f39c31f729e6fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:46 UTC) from {username='oryx', principal='oryx/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:39080
I20260806 14:58:46.017797 22397 tablet_replica.cc:330] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0: stopping tablet replica
I20260806 14:58:46.017808 22400 tablet_replica.cc:330] T 18216016191f44ab83f39c31f729e6fc P e2751e00e4174c9ebcf59dd0ea26cb94: stopping tablet replica
I20260806 14:58:46.017868 22397 raft_consensus.cc:2229] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0 [term 1 LEADER]: Raft consensus shutting down.
I20260806 14:58:46.017875 22400 raft_consensus.cc:2229] T 18216016191f44ab83f39c31f729e6fc P e2751e00e4174c9ebcf59dd0ea26cb94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:58:46.017918 22400 raft_consensus.cc:2258] T 18216016191f44ab83f39c31f729e6fc P e2751e00e4174c9ebcf59dd0ea26cb94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:46.017968 22397 raft_consensus.cc:2258] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:46.018107 22221 tablet_service.cc:1514] Processing DeleteTablet for tablet 18216016191f44ab83f39c31f729e6fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:46 UTC) from {username='oryx', principal='oryx/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:42396
I20260806 14:58:46.018203 22398 tablet_replica.cc:330] T 18216016191f44ab83f39c31f729e6fc P 37ad3fe8e68b4d488ef0b44d31584401: stopping tablet replica
I20260806 14:58:46.018256 22398 raft_consensus.cc:2229] T 18216016191f44ab83f39c31f729e6fc P 37ad3fe8e68b4d488ef0b44d31584401 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:58:46.018301 22398 raft_consensus.cc:2258] T 18216016191f44ab83f39c31f729e6fc P 37ad3fe8e68b4d488ef0b44d31584401 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:46.018339 22397 ts_tablet_manager.cc:1905] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:46.020109 22397 ts_tablet_manager.cc:1918] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 14:58:46.020166 22397 log.cc:1196] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-0/wal/wals/18216016191f44ab83f39c31f729e6fc
I20260806 14:58:46.020399 22397 ts_tablet_manager.cc:1939] T 18216016191f44ab83f39c31f729e6fc P 4c9b8da547274fca8cbca18abfebc2d0: Deleting consensus metadata
I20260806 14:58:46.021497 21594 catalog_manager.cc:4841] TS 4c9b8da547274fca8cbca18abfebc2d0 (127.11.8.65:34101): tablet 18216016191f44ab83f39c31f729e6fc (table test-table [id=b2a79582d40c44969762190bc27b7d6d]) successfully deleted
I20260806 14:58:46.022260 22400 ts_tablet_manager.cc:1905] T 18216016191f44ab83f39c31f729e6fc P e2751e00e4174c9ebcf59dd0ea26cb94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:46.022451 22398 ts_tablet_manager.cc:1905] T 18216016191f44ab83f39c31f729e6fc P 37ad3fe8e68b4d488ef0b44d31584401: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:46.024096 22400 ts_tablet_manager.cc:1918] T 18216016191f44ab83f39c31f729e6fc P e2751e00e4174c9ebcf59dd0ea26cb94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 14:58:46.024175 22400 log.cc:1196] T 18216016191f44ab83f39c31f729e6fc P e2751e00e4174c9ebcf59dd0ea26cb94: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-1/wal/wals/18216016191f44ab83f39c31f729e6fc
I20260806 14:58:46.024231 22398 ts_tablet_manager.cc:1918] T 18216016191f44ab83f39c31f729e6fc P 37ad3fe8e68b4d488ef0b44d31584401: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 14:58:46.024307 22398 log.cc:1196] T 18216016191f44ab83f39c31f729e6fc P 37ad3fe8e68b4d488ef0b44d31584401: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-2/wal/wals/18216016191f44ab83f39c31f729e6fc
I20260806 14:58:46.024410 22400 ts_tablet_manager.cc:1939] T 18216016191f44ab83f39c31f729e6fc P e2751e00e4174c9ebcf59dd0ea26cb94: Deleting consensus metadata
I20260806 14:58:46.024574 22398 ts_tablet_manager.cc:1939] T 18216016191f44ab83f39c31f729e6fc P 37ad3fe8e68b4d488ef0b44d31584401: Deleting consensus metadata
I20260806 14:58:46.025133 21589 catalog_manager.cc:4841] TS 37ad3fe8e68b4d488ef0b44d31584401 (127.11.8.67:40909): tablet 18216016191f44ab83f39c31f729e6fc (table test-table [id=b2a79582d40c44969762190bc27b7d6d]) successfully deleted
I20260806 14:58:46.025219 21589 catalog_manager.cc:4841] TS e2751e00e4174c9ebcf59dd0ea26cb94 (127.11.8.66:36205): tablet 18216016191f44ab83f39c31f729e6fc (table test-table [id=b2a79582d40c44969762190bc27b7d6d]) successfully deleted
Aug 06 14:58:46 dist-test-slave-68xk krb5kdc[21554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 14:58:46 dist-test-slave-68xk krb5kdc[21554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:46 dist-test-slave-68xk krb5kdc[21554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 14:58:46.048748 21685 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:46.041859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38320 (local address 127.11.8.126:36493)
0806 14:58:46.041916 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:46.041918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:46.042127 (+ 209us) server_negotiation.cc:407] Connection header received
0806 14:58:46.042195 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:46.042197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:46.042213 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:46.042278 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:46.043144 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:46.043701 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:46.044868 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:46.045071 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:46.045131 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:46.048665 (+ 3534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38320: BlockingRecv error: recv got EOF from 127.0.0.1:38320 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 06 14:58:46 dist-test-slave-68xk krb5kdc[21554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM
I20260806 14:58:46.058121 21685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:46.049895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38326 (local address 127.11.8.126:36493)
0806 14:58:46.049939 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:46.049941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:46.050031 (+ 90us) server_negotiation.cc:407] Connection header received
0806 14:58:46.050065 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:46.050066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:46.050078 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:46.050150 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:46.050685 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:46.051517 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:46.052185 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:46.052521 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:46.052631 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:46.055127 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:46.055129 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:46.055129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:46.055139 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:46.056964 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:46.057316 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:46.057317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:46.057318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:46.057389 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:46.057591 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:46.057593 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:46.057593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:46.057741 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:46.057852 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:46.057933 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 14:58:46.058021 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 14:58:46.059863 21615 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38326:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 14:58:46.060055 21615 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 14:58:46.061564 22007 tablet_service.cc:1467] Processing CreateTablet for tablet 689cb6185f154e14a04de4a2ef14efe6 (DEFAULT_TABLE table=test-table [id=9ee69533994f47938a9fd9734426d97f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:46.061615 22221 tablet_service.cc:1467] Processing CreateTablet for tablet 689cb6185f154e14a04de4a2ef14efe6 (DEFAULT_TABLE table=test-table [id=9ee69533994f47938a9fd9734426d97f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:46.061617 21810 tablet_service.cc:1467] Processing CreateTablet for tablet 689cb6185f154e14a04de4a2ef14efe6 (DEFAULT_TABLE table=test-table [id=9ee69533994f47938a9fd9734426d97f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:46.061717 22007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 689cb6185f154e14a04de4a2ef14efe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:46.061734 22221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 689cb6185f154e14a04de4a2ef14efe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:46.061764 21810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 689cb6185f154e14a04de4a2ef14efe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:46.063138 22365 tablet_bootstrap.cc:492] T 689cb6185f154e14a04de4a2ef14efe6 P e2751e00e4174c9ebcf59dd0ea26cb94: Bootstrap starting.
I20260806 14:58:46.063290 22362 tablet_bootstrap.cc:492] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0: Bootstrap starting.
I20260806 14:58:46.063345 22372 tablet_bootstrap.cc:492] T 689cb6185f154e14a04de4a2ef14efe6 P 37ad3fe8e68b4d488ef0b44d31584401: Bootstrap starting.
I20260806 14:58:46.063529 22365 tablet_bootstrap.cc:654] T 689cb6185f154e14a04de4a2ef14efe6 P e2751e00e4174c9ebcf59dd0ea26cb94: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:46.063764 22372 tablet_bootstrap.cc:654] T 689cb6185f154e14a04de4a2ef14efe6 P 37ad3fe8e68b4d488ef0b44d31584401: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:46.063833 22362 tablet_bootstrap.cc:654] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:46.064426 22372 tablet_bootstrap.cc:492] T 689cb6185f154e14a04de4a2ef14efe6 P 37ad3fe8e68b4d488ef0b44d31584401: No bootstrap required, opened a new log
I20260806 14:58:46.064502 22372 ts_tablet_manager.cc:1397] T 689cb6185f154e14a04de4a2ef14efe6 P 37ad3fe8e68b4d488ef0b44d31584401: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 14:58:46.064671 22372 raft_consensus.cc:348] T 689cb6185f154e14a04de4a2ef14efe6 P 37ad3fe8e68b4d488ef0b44d31584401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:46.064739 22362 tablet_bootstrap.cc:492] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0: No bootstrap required, opened a new log
I20260806 14:58:46.064760 22372 raft_consensus.cc:374] T 689cb6185f154e14a04de4a2ef14efe6 P 37ad3fe8e68b4d488ef0b44d31584401 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:46.064795 22372 raft_consensus.cc:729] T 689cb6185f154e14a04de4a2ef14efe6 P 37ad3fe8e68b4d488ef0b44d31584401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37ad3fe8e68b4d488ef0b44d31584401, State: Initialized, Role: FOLLOWER
I20260806 14:58:46.064788 22362 ts_tablet_manager.cc:1397] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 14:58:46.064837 22372 consensus_queue.cc:260] T 689cb6185f154e14a04de4a2ef14efe6 P 37ad3fe8e68b4d488ef0b44d31584401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:46.064930 22372 ts_tablet_manager.cc:1428] T 689cb6185f154e14a04de4a2ef14efe6 P 37ad3fe8e68b4d488ef0b44d31584401: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 14:58:46.064949 22362 raft_consensus.cc:348] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:46.065055 22362 raft_consensus.cc:374] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:46.065081 22362 raft_consensus.cc:729] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c9b8da547274fca8cbca18abfebc2d0, State: Initialized, Role: FOLLOWER
I20260806 14:58:46.065132 22362 consensus_queue.cc:260] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:46.065233 22362 ts_tablet_manager.cc:1428] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 14:58:46.065961 22365 tablet_bootstrap.cc:492] T 689cb6185f154e14a04de4a2ef14efe6 P e2751e00e4174c9ebcf59dd0ea26cb94: No bootstrap required, opened a new log
I20260806 14:58:46.066008 22365 ts_tablet_manager.cc:1397] T 689cb6185f154e14a04de4a2ef14efe6 P e2751e00e4174c9ebcf59dd0ea26cb94: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260806 14:58:46.066125 22365 raft_consensus.cc:348] T 689cb6185f154e14a04de4a2ef14efe6 P e2751e00e4174c9ebcf59dd0ea26cb94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:46.066195 22365 raft_consensus.cc:374] T 689cb6185f154e14a04de4a2ef14efe6 P e2751e00e4174c9ebcf59dd0ea26cb94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:46.066217 22365 raft_consensus.cc:729] T 689cb6185f154e14a04de4a2ef14efe6 P e2751e00e4174c9ebcf59dd0ea26cb94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2751e00e4174c9ebcf59dd0ea26cb94, State: Initialized, Role: FOLLOWER
I20260806 14:58:46.066272 22365 consensus_queue.cc:260] T 689cb6185f154e14a04de4a2ef14efe6 P e2751e00e4174c9ebcf59dd0ea26cb94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:46.066359 22365 ts_tablet_manager.cc:1428] T 689cb6185f154e14a04de4a2ef14efe6 P e2751e00e4174c9ebcf59dd0ea26cb94: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 14:58:46.077066 22411 raft_consensus.cc:482] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 14:58:46.077155 22411 raft_consensus.cc:504] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:46.077332 22411 leader_election.cc:288] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2751e00e4174c9ebcf59dd0ea26cb94 (127.11.8.66:36205), 37ad3fe8e68b4d488ef0b44d31584401 (127.11.8.67:40909)
I20260806 14:58:46.077589 22038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "689cb6185f154e14a04de4a2ef14efe6" candidate_uuid: "4c9b8da547274fca8cbca18abfebc2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" is_pre_election: true
I20260806 14:58:46.077574 22252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "689cb6185f154e14a04de4a2ef14efe6" candidate_uuid: "4c9b8da547274fca8cbca18abfebc2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37ad3fe8e68b4d488ef0b44d31584401" is_pre_election: true
I20260806 14:58:46.077695 22252 raft_consensus.cc:2454] T 689cb6185f154e14a04de4a2ef14efe6 P 37ad3fe8e68b4d488ef0b44d31584401 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c9b8da547274fca8cbca18abfebc2d0 in term 0.
I20260806 14:58:46.077710 22038 raft_consensus.cc:2454] T 689cb6185f154e14a04de4a2ef14efe6 P e2751e00e4174c9ebcf59dd0ea26cb94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c9b8da547274fca8cbca18abfebc2d0 in term 0.
I20260806 14:58:46.077886 21738 leader_election.cc:302] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c9b8da547274fca8cbca18abfebc2d0, e2751e00e4174c9ebcf59dd0ea26cb94; no voters:
I20260806 14:58:46.078008 22411 raft_consensus.cc:2781] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 14:58:46.078063 22411 raft_consensus.cc:482] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 14:58:46.078094 22411 raft_consensus.cc:3037] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:46.078910 22411 raft_consensus.cc:504] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:46.079061 22411 leader_election.cc:288] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0 [CANDIDATE]: Term 1 election: Requested vote from peers e2751e00e4174c9ebcf59dd0ea26cb94 (127.11.8.66:36205), 37ad3fe8e68b4d488ef0b44d31584401 (127.11.8.67:40909)
I20260806 14:58:46.079223 22038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "689cb6185f154e14a04de4a2ef14efe6" candidate_uuid: "4c9b8da547274fca8cbca18abfebc2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94"
I20260806 14:58:46.079254 22252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "689cb6185f154e14a04de4a2ef14efe6" candidate_uuid: "4c9b8da547274fca8cbca18abfebc2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37ad3fe8e68b4d488ef0b44d31584401"
I20260806 14:58:46.079313 22038 raft_consensus.cc:3037] T 689cb6185f154e14a04de4a2ef14efe6 P e2751e00e4174c9ebcf59dd0ea26cb94 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:46.079327 22252 raft_consensus.cc:3037] T 689cb6185f154e14a04de4a2ef14efe6 P 37ad3fe8e68b4d488ef0b44d31584401 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:46.079970 22038 raft_consensus.cc:2454] T 689cb6185f154e14a04de4a2ef14efe6 P e2751e00e4174c9ebcf59dd0ea26cb94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c9b8da547274fca8cbca18abfebc2d0 in term 1.
I20260806 14:58:46.080130 22252 raft_consensus.cc:2454] T 689cb6185f154e14a04de4a2ef14efe6 P 37ad3fe8e68b4d488ef0b44d31584401 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c9b8da547274fca8cbca18abfebc2d0 in term 1.
I20260806 14:58:46.080152 21738 leader_election.cc:302] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c9b8da547274fca8cbca18abfebc2d0, e2751e00e4174c9ebcf59dd0ea26cb94; no voters:
I20260806 14:58:46.080269 22411 raft_consensus.cc:2781] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:46.080324 22411 raft_consensus.cc:686] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0 [term 1 LEADER]: Becoming Leader. State: Replica: 4c9b8da547274fca8cbca18abfebc2d0, State: Running, Role: LEADER
I20260806 14:58:46.080379 22411 consensus_queue.cc:237] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:46.080998 21615 catalog_manager.cc:5495] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c9b8da547274fca8cbca18abfebc2d0 (127.11.8.65). New cstate: current_term: 1 leader_uuid: "4c9b8da547274fca8cbca18abfebc2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } health_report { overall_health: UNKNOWN } } }
I20260806 14:58:46.090615 22356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:46.087809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39106 (local address 127.11.8.65:34101)
0806 14:58:46.087878 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:46.087880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:46.087890 (+ 10us) server_negotiation.cc:407] Connection header received
0806 14:58:46.087936 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:46.087937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:46.087948 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:46.088013 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 14:58:46.088224 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:46.088861 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:46.089767 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:46.090060 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:46.090161 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:46.090326 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 14:58:46.090363 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 14:58:46.090452 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:46.090510 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 14:58:46.090529 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 14:58:46.091431 22252 raft_consensus.cc:1261] T 689cb6185f154e14a04de4a2ef14efe6 P 37ad3fe8e68b4d488ef0b44d31584401 [term 1 FOLLOWER]: Refusing update from remote peer 4c9b8da547274fca8cbca18abfebc2d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:46.091467 22038 raft_consensus.cc:1261] T 689cb6185f154e14a04de4a2ef14efe6 P e2751e00e4174c9ebcf59dd0ea26cb94 [term 1 FOLLOWER]: Refusing update from remote peer 4c9b8da547274fca8cbca18abfebc2d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:46.091663 22411 consensus_queue.cc:1035] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:46.091799 22411 consensus_queue.cc:1035] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:46.094893 21611 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:58:46.094978 21611 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:58:46.095650 21611 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b79710c047b8421f88b764f7af7025c6: Sending DeleteTablet for 3 replicas of tablet 689cb6185f154e14a04de4a2ef14efe6
I20260806 14:58:46.095955 22221 tablet_service.cc:1514] Processing DeleteTablet for tablet 689cb6185f154e14a04de4a2ef14efe6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:46 UTC) from {username='oryx', principal='oryx/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:42396
I20260806 14:58:46.095963 21810 tablet_service.cc:1514] Processing DeleteTablet for tablet 689cb6185f154e14a04de4a2ef14efe6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:46 UTC) from {username='oryx', principal='oryx/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:39080
I20260806 14:58:46.096060 22398 tablet_replica.cc:330] T 689cb6185f154e14a04de4a2ef14efe6 P 37ad3fe8e68b4d488ef0b44d31584401: stopping tablet replica
I20260806 14:58:46.096063 22397 tablet_replica.cc:330] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0: stopping tablet replica
I20260806 14:58:46.096119 22398 raft_consensus.cc:2229] T 689cb6185f154e14a04de4a2ef14efe6 P 37ad3fe8e68b4d488ef0b44d31584401 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:58:46.096190 22397 raft_consensus.cc:2229] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0 [term 1 LEADER]: Raft consensus shutting down.
I20260806 14:58:46.096272 22398 raft_consensus.cc:2258] T 689cb6185f154e14a04de4a2ef14efe6 P 37ad3fe8e68b4d488ef0b44d31584401 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:46.096280 22397 raft_consensus.cc:2258] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:46.096494 22007 tablet_service.cc:1514] Processing DeleteTablet for tablet 689cb6185f154e14a04de4a2ef14efe6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:46 UTC) from {username='oryx', principal='oryx/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:58284
I20260806 14:58:46.096494 21611 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38326:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 14:58:46.096586 22400 tablet_replica.cc:330] T 689cb6185f154e14a04de4a2ef14efe6 P e2751e00e4174c9ebcf59dd0ea26cb94: stopping tablet replica
W20260806 14:58:46.096644 21611 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 14:58:46.096655 22400 raft_consensus.cc:2229] T 689cb6185f154e14a04de4a2ef14efe6 P e2751e00e4174c9ebcf59dd0ea26cb94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:58:46.096659 22397 ts_tablet_manager.cc:1905] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:46.096696 22400 raft_consensus.cc:2258] T 689cb6185f154e14a04de4a2ef14efe6 P e2751e00e4174c9ebcf59dd0ea26cb94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:46.097185 22400 ts_tablet_manager.cc:1905] T 689cb6185f154e14a04de4a2ef14efe6 P e2751e00e4174c9ebcf59dd0ea26cb94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:46.098381 22397 ts_tablet_manager.cc:1918] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 14:58:46.098480 22397 log.cc:1196] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-0/wal/wals/689cb6185f154e14a04de4a2ef14efe6
I20260806 14:58:46.098702 22400 ts_tablet_manager.cc:1918] T 689cb6185f154e14a04de4a2ef14efe6 P e2751e00e4174c9ebcf59dd0ea26cb94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 14:58:46.098773 22400 log.cc:1196] T 689cb6185f154e14a04de4a2ef14efe6 P e2751e00e4174c9ebcf59dd0ea26cb94: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-1/wal/wals/689cb6185f154e14a04de4a2ef14efe6
I20260806 14:58:46.098788 22397 ts_tablet_manager.cc:1939] T 689cb6185f154e14a04de4a2ef14efe6 P 4c9b8da547274fca8cbca18abfebc2d0: Deleting consensus metadata
I20260806 14:58:46.099001 22400 ts_tablet_manager.cc:1939] T 689cb6185f154e14a04de4a2ef14efe6 P e2751e00e4174c9ebcf59dd0ea26cb94: Deleting consensus metadata
I20260806 14:58:46.099210 21594 catalog_manager.cc:4841] TS 4c9b8da547274fca8cbca18abfebc2d0 (127.11.8.65:34101): tablet 689cb6185f154e14a04de4a2ef14efe6 (table test-table [id=9ee69533994f47938a9fd9734426d97f]) successfully deleted
I20260806 14:58:46.099538 21589 catalog_manager.cc:4841] TS e2751e00e4174c9ebcf59dd0ea26cb94 (127.11.8.66:36205): tablet 689cb6185f154e14a04de4a2ef14efe6 (table test-table [id=9ee69533994f47938a9fd9734426d97f]) successfully deleted
I20260806 14:58:46.099680 22398 ts_tablet_manager.cc:1905] T 689cb6185f154e14a04de4a2ef14efe6 P 37ad3fe8e68b4d488ef0b44d31584401: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:46.099977 21810 tablet_service.cc:1467] Processing CreateTablet for tablet e9588cd5e70441b8ba8983d4cc5e11da (DEFAULT_TABLE table=test-table [id=6868877fd4064a59a437fda8db4a0845]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:46.099992 22221 tablet_service.cc:1467] Processing CreateTablet for tablet e9588cd5e70441b8ba8983d4cc5e11da (DEFAULT_TABLE table=test-table [id=6868877fd4064a59a437fda8db4a0845]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:46.100091 21810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9588cd5e70441b8ba8983d4cc5e11da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:46.100113 22221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9588cd5e70441b8ba8983d4cc5e11da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:46.101325 22398 ts_tablet_manager.cc:1918] T 689cb6185f154e14a04de4a2ef14efe6 P 37ad3fe8e68b4d488ef0b44d31584401: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 14:58:46.101405 22398 log.cc:1196] T 689cb6185f154e14a04de4a2ef14efe6 P 37ad3fe8e68b4d488ef0b44d31584401: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-2/wal/wals/689cb6185f154e14a04de4a2ef14efe6
I20260806 14:58:46.101464 22362 tablet_bootstrap.cc:492] T e9588cd5e70441b8ba8983d4cc5e11da P 4c9b8da547274fca8cbca18abfebc2d0: Bootstrap starting.
I20260806 14:58:46.101485 22372 tablet_bootstrap.cc:492] T e9588cd5e70441b8ba8983d4cc5e11da P 37ad3fe8e68b4d488ef0b44d31584401: Bootstrap starting.
I20260806 14:58:46.101667 22398 ts_tablet_manager.cc:1939] T 689cb6185f154e14a04de4a2ef14efe6 P 37ad3fe8e68b4d488ef0b44d31584401: Deleting consensus metadata
I20260806 14:58:46.101784 22362 tablet_bootstrap.cc:654] T e9588cd5e70441b8ba8983d4cc5e11da P 4c9b8da547274fca8cbca18abfebc2d0: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:46.101946 22372 tablet_bootstrap.cc:654] T e9588cd5e70441b8ba8983d4cc5e11da P 37ad3fe8e68b4d488ef0b44d31584401: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:46.102437 21589 catalog_manager.cc:4841] TS 37ad3fe8e68b4d488ef0b44d31584401 (127.11.8.67:40909): tablet 689cb6185f154e14a04de4a2ef14efe6 (table test-table [id=9ee69533994f47938a9fd9734426d97f]) successfully deleted
I20260806 14:58:46.102526 22362 tablet_bootstrap.cc:492] T e9588cd5e70441b8ba8983d4cc5e11da P 4c9b8da547274fca8cbca18abfebc2d0: No bootstrap required, opened a new log
I20260806 14:58:46.102595 22362 ts_tablet_manager.cc:1397] T e9588cd5e70441b8ba8983d4cc5e11da P 4c9b8da547274fca8cbca18abfebc2d0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 14:58:46.102653 22007 tablet_service.cc:1467] Processing CreateTablet for tablet e9588cd5e70441b8ba8983d4cc5e11da (DEFAULT_TABLE table=test-table [id=6868877fd4064a59a437fda8db4a0845]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:58:46.102754 22007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9588cd5e70441b8ba8983d4cc5e11da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:46.102708 22362 raft_consensus.cc:348] T e9588cd5e70441b8ba8983d4cc5e11da P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:46.102782 22362 raft_consensus.cc:374] T e9588cd5e70441b8ba8983d4cc5e11da P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:46.102804 22362 raft_consensus.cc:729] T e9588cd5e70441b8ba8983d4cc5e11da P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c9b8da547274fca8cbca18abfebc2d0, State: Initialized, Role: FOLLOWER
I20260806 14:58:46.102860 22362 consensus_queue.cc:260] T e9588cd5e70441b8ba8983d4cc5e11da P 4c9b8da547274fca8cbca18abfebc2d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:46.102957 22362 ts_tablet_manager.cc:1428] T e9588cd5e70441b8ba8983d4cc5e11da P 4c9b8da547274fca8cbca18abfebc2d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 14:58:46.103420 22372 tablet_bootstrap.cc:492] T e9588cd5e70441b8ba8983d4cc5e11da P 37ad3fe8e68b4d488ef0b44d31584401: No bootstrap required, opened a new log
I20260806 14:58:46.103473 22372 ts_tablet_manager.cc:1397] T e9588cd5e70441b8ba8983d4cc5e11da P 37ad3fe8e68b4d488ef0b44d31584401: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 14:58:46.103633 22372 raft_consensus.cc:348] T e9588cd5e70441b8ba8983d4cc5e11da P 37ad3fe8e68b4d488ef0b44d31584401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:46.103698 22372 raft_consensus.cc:374] T e9588cd5e70441b8ba8983d4cc5e11da P 37ad3fe8e68b4d488ef0b44d31584401 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:46.103721 22372 raft_consensus.cc:729] T e9588cd5e70441b8ba8983d4cc5e11da P 37ad3fe8e68b4d488ef0b44d31584401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37ad3fe8e68b4d488ef0b44d31584401, State: Initialized, Role: FOLLOWER
I20260806 14:58:46.103765 22372 consensus_queue.cc:260] T e9588cd5e70441b8ba8983d4cc5e11da P 37ad3fe8e68b4d488ef0b44d31584401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:46.103850 22372 ts_tablet_manager.cc:1428] T e9588cd5e70441b8ba8983d4cc5e11da P 37ad3fe8e68b4d488ef0b44d31584401: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:46.103871 22365 tablet_bootstrap.cc:492] T e9588cd5e70441b8ba8983d4cc5e11da P e2751e00e4174c9ebcf59dd0ea26cb94: Bootstrap starting.
I20260806 14:58:46.104216 22365 tablet_bootstrap.cc:654] T e9588cd5e70441b8ba8983d4cc5e11da P e2751e00e4174c9ebcf59dd0ea26cb94: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:46.104869 22365 tablet_bootstrap.cc:492] T e9588cd5e70441b8ba8983d4cc5e11da P e2751e00e4174c9ebcf59dd0ea26cb94: No bootstrap required, opened a new log
I20260806 14:58:46.104948 22365 ts_tablet_manager.cc:1397] T e9588cd5e70441b8ba8983d4cc5e11da P e2751e00e4174c9ebcf59dd0ea26cb94: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:46.105079 22365 raft_consensus.cc:348] T e9588cd5e70441b8ba8983d4cc5e11da P e2751e00e4174c9ebcf59dd0ea26cb94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:46.105172 22365 raft_consensus.cc:374] T e9588cd5e70441b8ba8983d4cc5e11da P e2751e00e4174c9ebcf59dd0ea26cb94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:46.105197 22365 raft_consensus.cc:729] T e9588cd5e70441b8ba8983d4cc5e11da P e2751e00e4174c9ebcf59dd0ea26cb94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2751e00e4174c9ebcf59dd0ea26cb94, State: Initialized, Role: FOLLOWER
I20260806 14:58:46.105247 22365 consensus_queue.cc:260] T e9588cd5e70441b8ba8983d4cc5e11da P e2751e00e4174c9ebcf59dd0ea26cb94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:46.105355 22365 ts_tablet_manager.cc:1428] T e9588cd5e70441b8ba8983d4cc5e11da P e2751e00e4174c9ebcf59dd0ea26cb94: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:46.271764 22411 raft_consensus.cc:482] T e9588cd5e70441b8ba8983d4cc5e11da P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 14:58:46.271889 22411 raft_consensus.cc:504] T e9588cd5e70441b8ba8983d4cc5e11da P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:46.272053 22411 leader_election.cc:288] T e9588cd5e70441b8ba8983d4cc5e11da P 4c9b8da547274fca8cbca18abfebc2d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2751e00e4174c9ebcf59dd0ea26cb94 (127.11.8.66:36205), 37ad3fe8e68b4d488ef0b44d31584401 (127.11.8.67:40909)
I20260806 14:58:46.272425 22038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9588cd5e70441b8ba8983d4cc5e11da" candidate_uuid: "4c9b8da547274fca8cbca18abfebc2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" is_pre_election: true
I20260806 14:58:46.272454 22252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9588cd5e70441b8ba8983d4cc5e11da" candidate_uuid: "4c9b8da547274fca8cbca18abfebc2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37ad3fe8e68b4d488ef0b44d31584401" is_pre_election: true
I20260806 14:58:46.272538 22252 raft_consensus.cc:2454] T e9588cd5e70441b8ba8983d4cc5e11da P 37ad3fe8e68b4d488ef0b44d31584401 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c9b8da547274fca8cbca18abfebc2d0 in term 0.
I20260806 14:58:46.272538 22038 raft_consensus.cc:2454] T e9588cd5e70441b8ba8983d4cc5e11da P e2751e00e4174c9ebcf59dd0ea26cb94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c9b8da547274fca8cbca18abfebc2d0 in term 0.
I20260806 14:58:46.272768 21738 leader_election.cc:302] T e9588cd5e70441b8ba8983d4cc5e11da P 4c9b8da547274fca8cbca18abfebc2d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37ad3fe8e68b4d488ef0b44d31584401, 4c9b8da547274fca8cbca18abfebc2d0; no voters:
I20260806 14:58:46.272886 22411 raft_consensus.cc:2781] T e9588cd5e70441b8ba8983d4cc5e11da P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 14:58:46.272933 22411 raft_consensus.cc:482] T e9588cd5e70441b8ba8983d4cc5e11da P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 14:58:46.272953 22411 raft_consensus.cc:3037] T e9588cd5e70441b8ba8983d4cc5e11da P 4c9b8da547274fca8cbca18abfebc2d0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:46.273689 22411 raft_consensus.cc:504] T e9588cd5e70441b8ba8983d4cc5e11da P 4c9b8da547274fca8cbca18abfebc2d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:46.273830 22411 leader_election.cc:288] T e9588cd5e70441b8ba8983d4cc5e11da P 4c9b8da547274fca8cbca18abfebc2d0 [CANDIDATE]: Term 1 election: Requested vote from peers e2751e00e4174c9ebcf59dd0ea26cb94 (127.11.8.66:36205), 37ad3fe8e68b4d488ef0b44d31584401 (127.11.8.67:40909)
I20260806 14:58:46.274034 22038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9588cd5e70441b8ba8983d4cc5e11da" candidate_uuid: "4c9b8da547274fca8cbca18abfebc2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94"
I20260806 14:58:46.274040 22252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9588cd5e70441b8ba8983d4cc5e11da" candidate_uuid: "4c9b8da547274fca8cbca18abfebc2d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37ad3fe8e68b4d488ef0b44d31584401"
I20260806 14:58:46.274127 22038 raft_consensus.cc:3037] T e9588cd5e70441b8ba8983d4cc5e11da P e2751e00e4174c9ebcf59dd0ea26cb94 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:46.274152 22252 raft_consensus.cc:3037] T e9588cd5e70441b8ba8983d4cc5e11da P 37ad3fe8e68b4d488ef0b44d31584401 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:46.274873 22038 raft_consensus.cc:2454] T e9588cd5e70441b8ba8983d4cc5e11da P e2751e00e4174c9ebcf59dd0ea26cb94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c9b8da547274fca8cbca18abfebc2d0 in term 1.
I20260806 14:58:46.274896 22252 raft_consensus.cc:2454] T e9588cd5e70441b8ba8983d4cc5e11da P 37ad3fe8e68b4d488ef0b44d31584401 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c9b8da547274fca8cbca18abfebc2d0 in term 1.
I20260806 14:58:46.275080 21738 leader_election.cc:302] T e9588cd5e70441b8ba8983d4cc5e11da P 4c9b8da547274fca8cbca18abfebc2d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c9b8da547274fca8cbca18abfebc2d0, e2751e00e4174c9ebcf59dd0ea26cb94; no voters:
I20260806 14:58:46.275218 22411 raft_consensus.cc:2781] T e9588cd5e70441b8ba8983d4cc5e11da P 4c9b8da547274fca8cbca18abfebc2d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:46.275286 22411 raft_consensus.cc:686] T e9588cd5e70441b8ba8983d4cc5e11da P 4c9b8da547274fca8cbca18abfebc2d0 [term 1 LEADER]: Becoming Leader. State: Replica: 4c9b8da547274fca8cbca18abfebc2d0, State: Running, Role: LEADER
I20260806 14:58:46.275336 22411 consensus_queue.cc:237] T e9588cd5e70441b8ba8983d4cc5e11da P 4c9b8da547274fca8cbca18abfebc2d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } }
I20260806 14:58:46.275983 21615 catalog_manager.cc:5495] T e9588cd5e70441b8ba8983d4cc5e11da P 4c9b8da547274fca8cbca18abfebc2d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c9b8da547274fca8cbca18abfebc2d0 (127.11.8.65). New cstate: current_term: 1 leader_uuid: "4c9b8da547274fca8cbca18abfebc2d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9b8da547274fca8cbca18abfebc2d0" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 } health_report { overall_health: UNKNOWN } } }
I20260806 14:58:46.285249 22252 raft_consensus.cc:1261] T e9588cd5e70441b8ba8983d4cc5e11da P 37ad3fe8e68b4d488ef0b44d31584401 [term 1 FOLLOWER]: Refusing update from remote peer 4c9b8da547274fca8cbca18abfebc2d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:46.285272 22038 raft_consensus.cc:1261] T e9588cd5e70441b8ba8983d4cc5e11da P e2751e00e4174c9ebcf59dd0ea26cb94 [term 1 FOLLOWER]: Refusing update from remote peer 4c9b8da547274fca8cbca18abfebc2d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:58:46.285478 22436 consensus_queue.cc:1035] T e9588cd5e70441b8ba8983d4cc5e11da P 4c9b8da547274fca8cbca18abfebc2d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2751e00e4174c9ebcf59dd0ea26cb94" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:46.285637 22411 consensus_queue.cc:1035] T e9588cd5e70441b8ba8983d4cc5e11da P 4c9b8da547274fca8cbca18abfebc2d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37ad3fe8e68b4d488ef0b44d31584401" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:58:46.298071 21611 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:58:46.298157 21611 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:58:46.298746 21611 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b79710c047b8421f88b764f7af7025c6: Sending DeleteTablet for 3 replicas of tablet e9588cd5e70441b8ba8983d4cc5e11da
I20260806 14:58:46.299000 22007 tablet_service.cc:1514] Processing DeleteTablet for tablet e9588cd5e70441b8ba8983d4cc5e11da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:46 UTC) from {username='oryx', principal='oryx/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:58284
I20260806 14:58:46.299019 22221 tablet_service.cc:1514] Processing DeleteTablet for tablet e9588cd5e70441b8ba8983d4cc5e11da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:58:46 UTC) from {username='oryx', principal='oryx/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:42396
I20260806 14:58:46.299101 22400 tablet_replica.cc:330] T e9588cd5e70441b8ba8983d4cc5e11da P e2751e00e4174c9ebcf59dd0ea26cb94: stopping tablet replica
I20260806 14:58:46.299121 22398 tablet_replica.cc:330] T e9588cd5e70441b8ba8983d4cc5e11da P 37ad3fe8e68b4d488ef0b44d31584401: stopping tablet replica
I20260806 14:58:46.299153 22400 raft_consensus.cc:2229] T e9588cd5e70441b8ba8983d4cc5e11da P e2751e00e4174c9ebcf59dd0ea26cb94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:58:46.299186 22398 raft_consensus.cc:2229] T e9588cd5e70441b8ba8983d4cc5e11da P 37ad3fe8e68b4d488ef0b44d31584401 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:58:46.299196 22400 raft_consensus.cc:2258] T e9588cd5e70441b8ba8983d4cc5e11da P e2751e00e4174c9ebcf59dd0ea26cb94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:46.299229 22398 raft_consensus.cc:2258] T e9588cd5e70441b8ba8983d4cc5e11da P 37ad3fe8e68b4d488ef0b44d31584401 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:58:46.299381 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 21712
I20260806 14:58:46.305781 22400 ts_tablet_manager.cc:1905] T e9588cd5e70441b8ba8983d4cc5e11da P e2751e00e4174c9ebcf59dd0ea26cb94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:46.307050 22398 ts_tablet_manager.cc:1905] T e9588cd5e70441b8ba8983d4cc5e11da P 37ad3fe8e68b4d488ef0b44d31584401: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:58:46.307415 22400 ts_tablet_manager.cc:1918] T e9588cd5e70441b8ba8983d4cc5e11da P e2751e00e4174c9ebcf59dd0ea26cb94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 14:58:46.307482 22400 log.cc:1196] T e9588cd5e70441b8ba8983d4cc5e11da P e2751e00e4174c9ebcf59dd0ea26cb94: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-1/wal/wals/e9588cd5e70441b8ba8983d4cc5e11da
I20260806 14:58:46.307703 22400 ts_tablet_manager.cc:1939] T e9588cd5e70441b8ba8983d4cc5e11da P e2751e00e4174c9ebcf59dd0ea26cb94: Deleting consensus metadata
I20260806 14:58:46.308238 21589 catalog_manager.cc:4841] TS e2751e00e4174c9ebcf59dd0ea26cb94 (127.11.8.66:36205): tablet e9588cd5e70441b8ba8983d4cc5e11da (table test-table [id=6868877fd4064a59a437fda8db4a0845]) successfully deleted
I20260806 14:58:46.308642 22398 ts_tablet_manager.cc:1918] T e9588cd5e70441b8ba8983d4cc5e11da P 37ad3fe8e68b4d488ef0b44d31584401: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 14:58:46.308694 22398 log.cc:1196] T e9588cd5e70441b8ba8983d4cc5e11da P 37ad3fe8e68b4d488ef0b44d31584401: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786028287744255-11297-0/minicluster-data/ts-2/wal/wals/e9588cd5e70441b8ba8983d4cc5e11da
I20260806 14:58:46.308847 22398 ts_tablet_manager.cc:1939] T e9588cd5e70441b8ba8983d4cc5e11da P 37ad3fe8e68b4d488ef0b44d31584401: Deleting consensus metadata
I20260806 14:58:46.309338 21589 catalog_manager.cc:4841] TS 37ad3fe8e68b4d488ef0b44d31584401 (127.11.8.67:40909): tablet e9588cd5e70441b8ba8983d4cc5e11da (table test-table [id=6868877fd4064a59a437fda8db4a0845]) successfully deleted
W20260806 14:58:46.315651 21594 connection.cc:535] client connection to 127.11.8.65:34101 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 14:58:46.315832 21594 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 14:58:46.316005 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 21918
I20260806 14:58:46.319654 22484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:46.318820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:34101 (local address 127.0.0.1:39120)
0806 14:58:46.319547 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:46.319603 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.65:34101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":706,"thread_start_us":659,"threads_started":1}
W20260806 14:58:46.322441 21594 catalog_manager.cc:4584] TS 4c9b8da547274fca8cbca18abfebc2d0 (127.11.8.65:34101): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e9588cd5e70441b8ba8983d4cc5e11da: Network error: Client connection negotiation failed: client connection to 127.11.8.65:34101: connect: Connection refused (error 111)
I20260806 14:58:46.328007 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 22127
I20260806 14:58:46.335255 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 21571
2026-08-06T14:58:46Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3068 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 14:58:46 dist-test-slave-68xk krb5kdc[22488](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 14:58:46 dist-test-slave-68xk krb5kdc[22488](info): set up 2 sockets
Aug 06 14:58:46 dist-test-slave-68xk krb5kdc[22488](info): commencing operation
krb5kdc: starting...
W20260806 14:58:47.577148 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 14:58:47 dist-test-slave-68xk krb5kdc[22488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T14:58:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T14:58:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 14:58:47.704835 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:39245
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:43375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:39245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:47.791335 22506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:47.791565 22506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:47.791589 22506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:47.792780 22506 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 14:58:47.792827 22506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:47.792841 22506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:47.792855 22506 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 14:58:47.792866 22506 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 14:58:47.794320 22506 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:43375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:39245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:39245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:47.794584 22506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:47.794765 22506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:47.799538 22516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:47.799546 22514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:47.799604 22513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:47.799670 22506 server_base.cc:1034] running on GCE node
I20260806 14:58:47.800024 22506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:47.800319 22506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:47.801461 22506 hybrid_clock.cc:648] HybridClock initialized: now 1786028327801452 us; error 29 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T14:58:47Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1471 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 14:58:47 dist-test-slave-68xk krb5kdc[22524](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 14:58:47 dist-test-slave-68xk krb5kdc[22524](info): set up 2 sockets
Aug 06 14:58:47 dist-test-slave-68xk krb5kdc[22524](info): commencing operation
krb5kdc: starting...
W20260806 14:58:49.023751 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 14:58:49 dist-test-slave-68xk krb5kdc[22524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T14:58:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T14:58:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 14:58:49.149809 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:45081
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:44007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:45081
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:49.232447 22539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:49.232637 22539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:49.232658 22539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:49.233755 22539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 14:58:49.233790 22539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:49.233804 22539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:49.233817 22539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 14:58:49.233829 22539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 14:58:49.235242 22539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:44007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:45081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:45081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:49.235430 22539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:49.235594 22539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:49.240422 22553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:49.240491 22556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:49.240502 22552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:49.240624 22539 server_base.cc:1034] running on GCE node
I20260806 14:58:49.240854 22539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:49.241086 22539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:49.242231 22539 hybrid_clock.cc:648] HybridClock initialized: now 1786028329242216 us; error 35 us; skew 500 ppm
Aug 06 14:58:49 dist-test-slave-68xk krb5kdc[22524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028329, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:49.251729 22539 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 14:58:49.252249 22539 webserver.cc:460] Webserver started at http://127.11.8.126:40183/ using document root <none> and password file <none>
I20260806 14:58:49.252507 22539 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:49.252579 22539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:49.252748 22539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:49.253844 22539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "d46692c87334416fa3be1855af3af8e1"
format_stamp: "Formatted at 2026-08-06 14:58:49 on dist-test-slave-68xk"
server_key: "9bcdb00bbe9f9760a3e358e80d45895a"
server_key_iv: "b0b0abd257c15a9882c61eeb35258674"
server_key_version: "encryptionkey@0"
I20260806 14:58:49.254221 22539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "d46692c87334416fa3be1855af3af8e1"
format_stamp: "Formatted at 2026-08-06 14:58:49 on dist-test-slave-68xk"
server_key: "9bcdb00bbe9f9760a3e358e80d45895a"
server_key_iv: "b0b0abd257c15a9882c61eeb35258674"
server_key_version: "encryptionkey@0"
I20260806 14:58:49.255661 22539 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 14:58:49.256486 22566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:49.256722 22539 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 14:58:49.256819 22539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "d46692c87334416fa3be1855af3af8e1"
format_stamp: "Formatted at 2026-08-06 14:58:49 on dist-test-slave-68xk"
server_key: "9bcdb00bbe9f9760a3e358e80d45895a"
server_key_iv: "b0b0abd257c15a9882c61eeb35258674"
server_key_version: "encryptionkey@0"
I20260806 14:58:49.256872 22539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:49.265266 22539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:49.267421 22539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:49.267566 22539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:49.272264 22539 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:45081
I20260806 14:58:49.272312 22648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:45081 every 8 connection(s)
I20260806 14:58:49.272704 22539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 14:58:49.273662 22649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:58:49.274842 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 22539
I20260806 14:58:49.274967 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 14:58:49.275156 11297 external_mini_cluster.cc:1428] Setting key b1e79a2194b5bd4a89c972c2276fa370
I20260806 14:58:49.276423 22649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1: Bootstrap starting.
I20260806 14:58:49.277036 22649 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1: Neither blocks nor log segments found. Creating new log.
I20260806 14:58:49.277431 22649 log.cc:824] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:49.278200 22649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1: No bootstrap required, opened a new log
I20260806 14:58:49.280448 22649 raft_consensus.cc:348] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46692c87334416fa3be1855af3af8e1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 45081 } }
I20260806 14:58:49.280607 22649 raft_consensus.cc:374] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:58:49.280658 22649 raft_consensus.cc:729] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d46692c87334416fa3be1855af3af8e1, State: Initialized, Role: FOLLOWER
I20260806 14:58:49.280810 22649 consensus_queue.cc:260] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46692c87334416fa3be1855af3af8e1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 45081 } }
I20260806 14:58:49.280879 22649 raft_consensus.cc:388] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 14:58:49.280901 22649 raft_consensus.cc:482] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 14:58:49.280963 22649 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:58:49.281515 22649 raft_consensus.cc:504] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46692c87334416fa3be1855af3af8e1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 45081 } }
I20260806 14:58:49.281603 22649 leader_election.cc:302] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d46692c87334416fa3be1855af3af8e1; no voters:
I20260806 14:58:49.281845 22649 leader_election.cc:288] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 14:58:49.281873 22657 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:58:49.281997 22657 raft_consensus.cc:686] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [term 1 LEADER]: Becoming Leader. State: Replica: d46692c87334416fa3be1855af3af8e1, State: Running, Role: LEADER
I20260806 14:58:49.282081 22657 consensus_queue.cc:237] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46692c87334416fa3be1855af3af8e1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 45081 } }
I20260806 14:58:49.282121 22649 sys_catalog.cc:564] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [sys.catalog]: configured and running, proceeding with master startup.
Aug 06 14:58:49 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:49.282469 22657 sys_catalog.cc:455] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d46692c87334416fa3be1855af3af8e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46692c87334416fa3be1855af3af8e1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 45081 } } }
I20260806 14:58:49.282644 22657 sys_catalog.cc:458] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:49.282862 22657 sys_catalog.cc:455] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d46692c87334416fa3be1855af3af8e1. Latest consensus state: current_term: 1 leader_uuid: "d46692c87334416fa3be1855af3af8e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46692c87334416fa3be1855af3af8e1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 45081 } } }
I20260806 14:58:49.282989 22657 sys_catalog.cc:458] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:49.283367 22671 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 14:58:49.283931 22671 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 14:58:49.285640 22671 catalog_manager.cc:1347] Generated new cluster ID: 7d01ba4d2a6c4c029d6d46fd0a9e988a
I20260806 14:58:49.285710 22671 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 14:58:49.287736 22653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:49.275722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50208 (local address 127.11.8.126:45081)
0806 14:58:49.275908 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:49.275916 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:49.275941 (+ 25us) server_negotiation.cc:407] Connection header received
0806 14:58:49.276359 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:49.276365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:49.276408 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:49.276568 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:49.277252 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:49.278624 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:49.279397 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:49.280110 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:49.280240 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:49.283047 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:49.283052 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:49.283053 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:49.283065 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:49.285965 (+ 2900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:49.286398 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:49.286402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:49.286402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:49.286571 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:49.286802 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:49.286803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:49.286804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:49.287075 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:49.287254 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:49.287376 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 14:58:49.287545 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":87,"threads_started":1}
I20260806 14:58:49.308823 22671 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 14:58:49.309558 22671 catalog_manager.cc:1504] Loading token signing keys...
I20260806 14:58:49.321691 22671 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1: Generated new TSK 0
I20260806 14:58:49.321903 22671 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
I20260806 14:58:49.357365 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:0
--local_ip_for_outbound_sockets=127.11.8.65
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:45081
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:44007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:49.440398 22689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:49.440595 22689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:49.440619 22689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:49.441710 22689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:49.441784 22689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:49.441833 22689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 14:58:49.443316 22689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:44007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:45081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:49.443500 22689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:49.443670 22689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:49.448669 22696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:49.448685 22694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:49.448815 22689 server_base.cc:1034] running on GCE node
W20260806 14:58:49.448665 22699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:49.449096 22689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:49.449347 22689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:49.450484 22689 hybrid_clock.cc:648] HybridClock initialized: now 1786028329450464 us; error 35 us; skew 500 ppm
Aug 06 14:58:49 dist-test-slave-68xk krb5kdc[22524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028329, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:49.453214 22689 init.cc:377] Logged in from keytab as kudu/127.11.8.65@KRBTEST.COM (short username kudu)
I20260806 14:58:49.453672 22689 webserver.cc:460] Webserver started at http://127.11.8.65:35659/ using document root <none> and password file <none>
I20260806 14:58:49.453884 22689 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:49.453925 22689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:49.454015 22689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:49.455052 22689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/data/instance:
uuid: "9bec7f475c1b45139cb3226390528929"
format_stamp: "Formatted at 2026-08-06 14:58:49 on dist-test-slave-68xk"
server_key: "86ceb3f73e74ffc52af45ee8c5ebf8d6"
server_key_iv: "5c06fe01840bfa1d50680da91b9a86b7"
server_key_version: "encryptionkey@0"
I20260806 14:58:49.455374 22689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance:
uuid: "9bec7f475c1b45139cb3226390528929"
format_stamp: "Formatted at 2026-08-06 14:58:49 on dist-test-slave-68xk"
server_key: "86ceb3f73e74ffc52af45ee8c5ebf8d6"
server_key_iv: "5c06fe01840bfa1d50680da91b9a86b7"
server_key_version: "encryptionkey@0"
I20260806 14:58:49.456626 22689 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 14:58:49.457350 22708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:49.457552 22689 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:49.457630 22689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "9bec7f475c1b45139cb3226390528929"
format_stamp: "Formatted at 2026-08-06 14:58:49 on dist-test-slave-68xk"
server_key: "86ceb3f73e74ffc52af45ee8c5ebf8d6"
server_key_iv: "5c06fe01840bfa1d50680da91b9a86b7"
server_key_version: "encryptionkey@0"
I20260806 14:58:49.457695 22689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:49.470520 22689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:49.472368 22689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:49.472478 22689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:49.472656 22689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:49.472982 22689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:49.473018 22689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:49.473039 22689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:49.473086 22689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:49.478919 22689 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:36477
I20260806 14:58:49.479012 22880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:36477 every 8 connection(s)
I20260806 14:58:49.479328 22689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
I20260806 14:58:49.482402 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 22689
I20260806 14:58:49.482537 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance
I20260806 14:58:49.482805 11297 external_mini_cluster.cc:1428] Setting key ace499dd145ed5ef00de74c2efc1d2fc
Aug 06 14:58:49 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028329, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:49.489697 22653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:49.480059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:48227 (local address 127.11.8.126:45081)
0806 14:58:49.480105 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:49.480107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:49.480310 (+ 203us) server_negotiation.cc:407] Connection header received
0806 14:58:49.480837 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:49.480839 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:49.480851 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:49.480917 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:49.481888 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:49.482618 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:49.484088 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:49.484302 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:49.484359 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:49.486804 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:49.486806 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:49.486807 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:49.486814 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:49.488282 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:49.488694 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:49.488696 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:49.488697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:49.488762 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:49.489024 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:49.489025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:49.489025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:49.489142 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:49.489227 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:49.489504 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 14:58:49.489606 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 14:58:49.489886 22883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:49.480029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.65:48227)
0806 14:58:49.480169 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:49.480222 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:49.480730 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:49.480983 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:49.480987 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:49.481571 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:49.481782 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:49.481787 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:49.482775 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:49.482777 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:49.484004 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:49.484007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:49.484390 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:49.484391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:49.484539 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:49.485176 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:49.485187 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:49.486705 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:49.488371 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:49.488379 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:49.488381 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:49.488599 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:49.488832 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:49.488832 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:49.488833 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:49.488946 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:49.489216 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:49.489224 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:49.489365 (+ 141us) client_negotiation.cc:769] Sending connection context
0806 14:58:49.489470 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 14:58:49.489625 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
I20260806 14:58:49.490383 22881 heartbeater.cc:347] Connected to a master server at 127.11.8.126:45081
I20260806 14:58:49.490520 22881 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:49.490743 22881 heartbeater.cc:511] Master 127.11.8.126:45081 requested a full tablet report, sending...
I20260806 14:58:49.491317 22580 ts_manager.cc:194] Registered new tserver with Master: 9bec7f475c1b45139cb3226390528929 (127.11.8.65:36477)
I20260806 14:58:49.491917 22580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.65@KRBTEST.COM'} at 127.11.8.65:48227
WARNING: no policy specified for kudu/127.11.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
I20260806 14:58:49.513266 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:0
--local_ip_for_outbound_sockets=127.11.8.66
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:45081
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:44007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:49.597847 22887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:49.598064 22887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:49.598090 22887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:49.599364 22887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:49.599403 22887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:49.599431 22887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 14:58:49.600924 22887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:44007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:45081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:49.601130 22887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:49.601317 22887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:49.606361 22897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:49.606303 22894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:49.606317 22895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:49.606652 22887 server_base.cc:1034] running on GCE node
I20260806 14:58:49.606846 22887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:49.607033 22887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:49.608219 22887 hybrid_clock.cc:648] HybridClock initialized: now 1786028329608200 us; error 28 us; skew 500 ppm
Aug 06 14:58:49 dist-test-slave-68xk krb5kdc[22524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028329, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:49.610885 22887 init.cc:377] Logged in from keytab as kudu/127.11.8.66@KRBTEST.COM (short username kudu)
I20260806 14:58:49.611385 22887 webserver.cc:460] Webserver started at http://127.11.8.66:41687/ using document root <none> and password file <none>
I20260806 14:58:49.611635 22887 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:49.611706 22887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:49.611860 22887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:49.612888 22887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/data/instance:
uuid: "3e7db4affa5f4c699e240398a8347a7c"
format_stamp: "Formatted at 2026-08-06 14:58:49 on dist-test-slave-68xk"
server_key: "c59d4804d3453a54d470511f15e4deb6"
server_key_iv: "0d6a2ab935e78d99eb3894125da0b367"
server_key_version: "encryptionkey@0"
I20260806 14:58:49.613265 22887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance:
uuid: "3e7db4affa5f4c699e240398a8347a7c"
format_stamp: "Formatted at 2026-08-06 14:58:49 on dist-test-slave-68xk"
server_key: "c59d4804d3453a54d470511f15e4deb6"
server_key_iv: "0d6a2ab935e78d99eb3894125da0b367"
server_key_version: "encryptionkey@0"
I20260806 14:58:49.614774 22887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 14:58:49.615506 22911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:49.615686 22887 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 14:58:49.615795 22887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "3e7db4affa5f4c699e240398a8347a7c"
format_stamp: "Formatted at 2026-08-06 14:58:49 on dist-test-slave-68xk"
server_key: "c59d4804d3453a54d470511f15e4deb6"
server_key_iv: "0d6a2ab935e78d99eb3894125da0b367"
server_key_version: "encryptionkey@0"
I20260806 14:58:49.615868 22887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:49.634541 22887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:49.636915 22887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:49.637094 22887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:49.637338 22887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:49.637645 22887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:49.637705 22887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:49.637746 22887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:49.637771 22887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:49.644105 22887 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:40519
I20260806 14:58:49.644201 23082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:40519 every 8 connection(s)
I20260806 14:58:49.644490 22887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
I20260806 14:58:49.648543 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 22887
I20260806 14:58:49.648680 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance
I20260806 14:58:49.648876 11297 external_mini_cluster.cc:1428] Setting key efb7622ef96f107efe5a7b353fcef49c
Aug 06 14:58:49 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028329, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:49.654287 22653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:49.645290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:37023 (local address 127.11.8.126:45081)
0806 14:58:49.645364 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:49.645366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:49.645567 (+ 201us) server_negotiation.cc:407] Connection header received
0806 14:58:49.646083 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:49.646084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:49.646094 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:49.646151 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:49.647143 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:49.647641 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:49.648647 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:49.648903 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:49.648980 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:49.651200 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:49.651201 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:49.651202 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:49.651210 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:49.652617 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:49.653107 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:49.653108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:49.653108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:49.653156 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:49.653475 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:49.653476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:49.653476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:49.653590 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:49.653649 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:49.654135 (+ 486us) server_negotiation.cc:299] Negotiation successful
0806 14:58:49.654224 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 14:58:49.654505 23089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:49.645257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.66:37023)
0806 14:58:49.645424 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:49.645466 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:49.645992 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:49.646209 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:49.646213 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:49.646875 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:49.647064 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:49.647069 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:49.647731 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:49.647731 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:49.648538 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:49.648541 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:49.648987 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:49.648989 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:49.649127 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:49.649668 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:49.649679 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:49.651102 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:49.652703 (+ 1601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:49.652710 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:49.652712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:49.653027 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:49.653232 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:49.653233 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:49.653233 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:49.653380 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:49.653663 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:49.653670 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:49.653870 (+ 200us) client_negotiation.cc:769] Sending connection context
0806 14:58:49.653990 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 14:58:49.654178 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
I20260806 14:58:49.655027 23086 heartbeater.cc:347] Connected to a master server at 127.11.8.126:45081
I20260806 14:58:49.655135 23086 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:49.655313 23086 heartbeater.cc:511] Master 127.11.8.126:45081 requested a full tablet report, sending...
I20260806 14:58:49.655730 22580 ts_manager.cc:194] Registered new tserver with Master: 3e7db4affa5f4c699e240398a8347a7c (127.11.8.66:40519)
I20260806 14:58:49.656292 22580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.66@KRBTEST.COM'} at 127.11.8.66:37023
WARNING: no policy specified for kudu/127.11.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
I20260806 14:58:49.677969 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:0
--local_ip_for_outbound_sockets=127.11.8.67
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:45081
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:44007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:49.762005 23092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:49.762223 23092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:49.762250 23092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:49.763447 23092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:49.763494 23092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:49.763520 23092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 14:58:49.764986 23092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:44007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:45081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:49.765267 23092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:49.765506 23092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:49.770694 23103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:49.770668 23106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:49.770664 23100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:49.770733 23092 server_base.cc:1034] running on GCE node
I20260806 14:58:49.771129 23092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:49.771376 23092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:49.772512 23092 hybrid_clock.cc:648] HybridClock initialized: now 1786028329772501 us; error 34 us; skew 500 ppm
Aug 06 14:58:49 dist-test-slave-68xk krb5kdc[22524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028329, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:49.775146 23092 init.cc:377] Logged in from keytab as kudu/127.11.8.67@KRBTEST.COM (short username kudu)
I20260806 14:58:49.775652 23092 webserver.cc:460] Webserver started at http://127.11.8.67:45835/ using document root <none> and password file <none>
I20260806 14:58:49.775909 23092 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:49.775982 23092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:49.776144 23092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:58:49.777197 23092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/data/instance:
uuid: "e237d7e262474e789170789129ce2cc4"
format_stamp: "Formatted at 2026-08-06 14:58:49 on dist-test-slave-68xk"
server_key: "54ecb0967708242405fd33a379266826"
server_key_iv: "58f39d4116f5e0d42049104a0e911bf7"
server_key_version: "encryptionkey@0"
I20260806 14:58:49.777552 23092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance:
uuid: "e237d7e262474e789170789129ce2cc4"
format_stamp: "Formatted at 2026-08-06 14:58:49 on dist-test-slave-68xk"
server_key: "54ecb0967708242405fd33a379266826"
server_key_iv: "58f39d4116f5e0d42049104a0e911bf7"
server_key_version: "encryptionkey@0"
I20260806 14:58:49.778965 23092 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 14:58:49.779670 23116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:49.779845 23092 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 14:58:49.779953 23092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "e237d7e262474e789170789129ce2cc4"
format_stamp: "Formatted at 2026-08-06 14:58:49 on dist-test-slave-68xk"
server_key: "54ecb0967708242405fd33a379266826"
server_key_iv: "58f39d4116f5e0d42049104a0e911bf7"
server_key_version: "encryptionkey@0"
I20260806 14:58:49.780026 23092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:49.816977 23092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:49.819262 23092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:49.819435 23092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:49.819693 23092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:49.820050 23092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:49.820111 23092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:49.820155 23092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:49.820183 23092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:49.826380 23092 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:46573
I20260806 14:58:49.826484 23292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:46573 every 8 connection(s)
I20260806 14:58:49.826834 23092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
Aug 06 14:58:49 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028329, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:58:49.833920 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 23092
I20260806 14:58:49.834023 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance
I20260806 14:58:49.834254 11297 external_mini_cluster.cc:1428] Setting key 7ec69abc5d220e0e2fd71989530c420c
I20260806 14:58:49.836689 22653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:49.827603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:49613 (local address 127.11.8.126:45081)
0806 14:58:49.827668 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:49.827671 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:49.827900 (+ 229us) server_negotiation.cc:407] Connection header received
0806 14:58:49.828368 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:49.828370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:49.828381 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:49.828435 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:49.829345 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:49.829843 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:49.830920 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:49.831110 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:49.831169 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:49.833318 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:49.833319 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:49.833320 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:49.833326 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:49.834857 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:49.835356 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:49.835357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:49.835358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:49.835428 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:49.835721 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:49.835722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:49.835723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:49.835856 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:49.835944 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:49.836475 (+ 531us) server_negotiation.cc:299] Negotiation successful
0806 14:58:49.836576 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 14:58:49.836875 23298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:49.827589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.67:49613)
0806 14:58:49.827772 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:49.827808 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:49.828283 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:49.828490 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:49.828493 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:49.829063 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:49.829264 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:49.829268 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:49.829928 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:49.829929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:49.830843 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:49.830846 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:49.831195 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:49.831195 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:49.831315 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:49.831771 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:49.831780 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:49.833215 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:58:49.834953 (+ 1738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:49.834959 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:49.834961 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:49.835264 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:49.835511 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:58:49.835512 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:58:49.835512 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:58:49.835643 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:58:49.835934 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:58:49.835941 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:58:49.836297 (+ 356us) client_negotiation.cc:769] Sending connection context
0806 14:58:49.836450 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 14:58:49.836620 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":120,"threads_started":1}
I20260806 14:58:49.837412 23294 heartbeater.cc:347] Connected to a master server at 127.11.8.126:45081
I20260806 14:58:49.837519 23294 heartbeater.cc:464] Registering TS with master...
I20260806 14:58:49.837704 23294 heartbeater.cc:511] Master 127.11.8.126:45081 requested a full tablet report, sending...
I20260806 14:58:49.838137 22580 ts_manager.cc:194] Registered new tserver with Master: e237d7e262474e789170789129ce2cc4 (127.11.8.67:46573)
I20260806 14:58:49.838534 22580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.67@KRBTEST.COM'} at 127.11.8.67:49613
I20260806 14:58:49.846601 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.11.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.65.keytab.
Entry for principal oryx/127.11.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.65.keytab.
WARNING: no policy specified for oryx/127.11.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.66.keytab.
Entry for principal oryx/127.11.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.66.keytab.
WARNING: no policy specified for oryx/127.11.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.11.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.11.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.67.keytab.
Entry for principal oryx/127.11.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.67.keytab.
I20260806 14:58:49.924564 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 22689
I20260806 14:58:49.937868 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 22887
I20260806 14:58:49.945005 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 23092
I20260806 14:58:49.954375 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 22539
I20260806 14:58:49.963107 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:45081
--webserver_interface=127.11.8.126
--webserver_port=40183
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:44007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:45081
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:58:50.046567 23310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:50.046773 23310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:50.046795 23310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:50.047968 23310 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 14:58:50.048014 23310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:50.048029 23310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:50.048043 23310 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 14:58:50.048101 23310 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 14:58:50.049571 23310 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:44007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:45081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:45081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=40183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:50.049834 23310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:50.050069 23310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:50.054955 23323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:50.054966 23318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:50.054960 23320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:50.055243 23310 server_base.cc:1034] running on GCE node
I20260806 14:58:50.055474 23310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:50.055712 23310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:50.056854 23310 hybrid_clock.cc:648] HybridClock initialized: now 1786028330056837 us; error 37 us; skew 500 ppm
Aug 06 14:58:50 dist-test-slave-68xk krb5kdc[22524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028330, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:50.059573 23310 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 14:58:50.060003 23310 webserver.cc:460] Webserver started at http://127.11.8.126:40183/ using document root <none> and password file <none>
I20260806 14:58:50.060262 23310 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:50.060357 23310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:50.061741 23310 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 14:58:50.062516 23336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:50.062697 23310 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 14:58:50.062803 23310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "d46692c87334416fa3be1855af3af8e1"
format_stamp: "Formatted at 2026-08-06 14:58:49 on dist-test-slave-68xk"
server_key: "9bcdb00bbe9f9760a3e358e80d45895a"
server_key_iv: "b0b0abd257c15a9882c61eeb35258674"
server_key_version: "encryptionkey@0"
I20260806 14:58:50.063177 23310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:50.071079 23310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:50.073028 23310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:50.073266 23310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:50.077603 23310 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:45081
I20260806 14:58:50.077630 23417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:45081 every 8 connection(s)
I20260806 14:58:50.078003 23310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 14:58:50.079499 23419 sys_catalog.cc:263] Verifying existing consensus state
I20260806 14:58:50.080137 23419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1: Bootstrap starting.
I20260806 14:58:50.088197 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 23310
I20260806 14:58:50.088541 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:36477
--local_ip_for_outbound_sockets=127.11.8.65
--tserver_master_addrs=127.11.8.126:45081
--webserver_port=35659
--webserver_interface=127.11.8.65
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:44007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.8.65
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 14:58:50.088542 23419 log.cc:824] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1: Log is configured to *not* fsync() on all Append() calls
I20260806 14:58:50.090651 23419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 14:58:50.090938 23419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1: Bootstrap complete.
I20260806 14:58:50.094741 23419 raft_consensus.cc:348] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46692c87334416fa3be1855af3af8e1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 45081 } }
I20260806 14:58:50.095129 23419 raft_consensus.cc:729] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d46692c87334416fa3be1855af3af8e1, State: Initialized, Role: FOLLOWER
I20260806 14:58:50.095332 23419 consensus_queue.cc:260] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46692c87334416fa3be1855af3af8e1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 45081 } }
I20260806 14:58:50.095422 23419 raft_consensus.cc:388] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 14:58:50.095480 23419 raft_consensus.cc:482] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 14:58:50.095541 23419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [term 1 FOLLOWER]: Advancing to term 2
I20260806 14:58:50.097067 23419 raft_consensus.cc:504] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46692c87334416fa3be1855af3af8e1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 45081 } }
I20260806 14:58:50.097204 23419 leader_election.cc:302] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d46692c87334416fa3be1855af3af8e1; no voters:
I20260806 14:58:50.097388 23419 leader_election.cc:288] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 14:58:50.097424 23427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 14:58:50.097616 23427 raft_consensus.cc:686] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [term 2 LEADER]: Becoming Leader. State: Replica: d46692c87334416fa3be1855af3af8e1, State: Running, Role: LEADER
I20260806 14:58:50.097692 23419 sys_catalog.cc:564] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 14:58:50.097745 23427 consensus_queue.cc:237] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46692c87334416fa3be1855af3af8e1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 45081 } }
I20260806 14:58:50.098166 23428 sys_catalog.cc:455] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d46692c87334416fa3be1855af3af8e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46692c87334416fa3be1855af3af8e1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 45081 } } }
I20260806 14:58:50.098165 23429 sys_catalog.cc:455] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d46692c87334416fa3be1855af3af8e1. Latest consensus state: current_term: 2 leader_uuid: "d46692c87334416fa3be1855af3af8e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46692c87334416fa3be1855af3af8e1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 45081 } } }
I20260806 14:58:50.098286 23428 sys_catalog.cc:458] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:50.098291 23429 sys_catalog.cc:458] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1 [sys.catalog]: This master's current role is: LEADER
I20260806 14:58:50.098680 23435 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 14:58:50.099345 23435 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 14:58:50.099519 23435 catalog_manager.cc:1259] Loaded cluster ID: 7d01ba4d2a6c4c029d6d46fd0a9e988a
I20260806 14:58:50.099720 23435 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 14:58:50.100878 23435 catalog_manager.cc:1504] Loading token signing keys...
I20260806 14:58:50.101049 23435 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d46692c87334416fa3be1855af3af8e1: Loaded TSK: 0
I20260806 14:58:50.101284 23435 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 14:58:50.173676 23425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:50.173925 23425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:50.173966 23425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:50.175141 23425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:50.175217 23425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:50.175263 23425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 14:58:50.176741 23425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:44007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.65.keytab
--principal=oryx/127.11.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:36477
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=35659
--tserver_master_addrs=127.11.8.126:45081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:50.177002 23425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:50.177237 23425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:50.183367 23461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:50.183459 23425 server_base.cc:1034] running on GCE node
W20260806 14:58:50.183367 23463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:50.183584 23459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:50.183881 23425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:50.184144 23425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:50.185364 23425 hybrid_clock.cc:648] HybridClock initialized: now 1786028330185339 us; error 39 us; skew 500 ppm
Aug 06 14:58:50 dist-test-slave-68xk krb5kdc[22524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028330, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:50.187947 23425 init.cc:377] Logged in from keytab as oryx/127.11.8.65@KRBTEST.COM (short username oryx)
I20260806 14:58:50.188411 23425 webserver.cc:460] Webserver started at http://127.11.8.65:35659/ using document root <none> and password file <none>
I20260806 14:58:50.188652 23425 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:50.188743 23425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:50.190166 23425 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:58:50.190917 23471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:50.191102 23425 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 14:58:50.191229 23425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "9bec7f475c1b45139cb3226390528929"
format_stamp: "Formatted at 2026-08-06 14:58:49 on dist-test-slave-68xk"
server_key: "86ceb3f73e74ffc52af45ee8c5ebf8d6"
server_key_iv: "5c06fe01840bfa1d50680da91b9a86b7"
server_key_version: "encryptionkey@0"
I20260806 14:58:50.191591 23425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:50.217540 23425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:50.220249 23425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:50.220436 23425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:50.220705 23425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:50.221020 23425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:50.221083 23425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:50.221129 23425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:50.221174 23425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:50.227789 23425 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:36477
I20260806 14:58:50.227842 23645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:36477 every 8 connection(s)
I20260806 14:58:50.228229 23425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
I20260806 14:58:50.235230 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 23425
Aug 06 14:58:50 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:50.235618 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:40519
--local_ip_for_outbound_sockets=127.11.8.66
--tserver_master_addrs=127.11.8.126:45081
--webserver_port=41687
--webserver_interface=127.11.8.66
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:44007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.8.66
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 14:58:50 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:50.236965 23648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:50.228981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.65:54403)
0806 14:58:50.229155 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:50.229215 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:50.229742 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:50.230191 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:50.230195 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:50.230858 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:50.231053 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:50.231057 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.232172 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.232173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:50.233025 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:50.233028 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.233693 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.233694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:50.233826 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:50.234390 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:50.234399 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:50.236747 (+ 2348us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":110,"threads_started":1}
W20260806 14:58:50.237011 23649 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:50.229041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:54403 (local address 127.11.8.126:45081)
0806 14:58:50.229285 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:50.229293 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:50.229337 (+ 44us) server_negotiation.cc:407] Connection header received
0806 14:58:50.229952 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:50.229960 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:50.230010 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:50.230155 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:50.231176 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.232039 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.233120 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.233535 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.233716 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:50.236866 (+ 3150us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.65:54403: BlockingRecv error: recv got EOF from 127.11.8.65:54403 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":130,"threads_started":1}
W20260806 14:58:50.237628 23646 heartbeater.cc:644] Failed to heartbeat to 127.11.8.126:45081 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.8.126:45081: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Aug 06 14:58:50 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:50 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:50.242210 23648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:50.237883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.65:53593)
0806 14:58:50.237992 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:50.238002 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:50.238040 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:50.238180 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:50.238180 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:50.238291 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:50.238334 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:50.238335 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.239006 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.239007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:50.239502 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:50.239504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.239912 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.239913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:50.239984 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:50.240452 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:50.240456 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:50.242117 (+ 1661us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":85}
W20260806 14:58:50.242242 23649 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:50.237940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:53593 (local address 127.11.8.126:45081)
0806 14:58:50.238014 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:50.238017 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:50.238030 (+ 13us) server_negotiation.cc:407] Connection header received
0806 14:58:50.238094 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:50.238095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:50.238109 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:50.238178 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:50.238405 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.238919 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.239627 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.239837 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.239913 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:50.242197 (+ 2284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.65:53593: BlockingRecv error: recv got EOF from 127.11.8.65:53593 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 14:58:50 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:50 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:50.247200 23648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:50.242479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.65:56773)
0806 14:58:50.242645 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:50.242656 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:50.242710 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:50.242832 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:50.242833 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:50.242989 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:50.243028 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:50.243030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.243620 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.243621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:50.244475 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:50.244478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.244858 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.244859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:50.244926 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:50.245319 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:50.245325 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:50.247110 (+ 1785us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":110,"threads_started":1}
W20260806 14:58:50.247234 23649 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:50.242516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:56773 (local address 127.11.8.126:45081)
0806 14:58:50.242546 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:50.242548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:50.242716 (+ 168us) server_negotiation.cc:407] Connection header received
0806 14:58:50.242756 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:50.242757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:50.242766 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:50.242806 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:50.243106 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.243538 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.244612 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.244781 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.244830 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:50.247176 (+ 2346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.65:56773: BlockingRecv error: recv got EOF from 127.11.8.65:56773 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260806 14:58:50.247350 23646 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 14:58:50.320511 23650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:50.320756 23650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:50.320796 23650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:50.321962 23650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:50.322031 23650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:50.322091 23650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 14:58:50.323592 23650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:44007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.66.keytab
--principal=oryx/127.11.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:40519
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=41687
--tserver_master_addrs=127.11.8.126:45081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:50.323864 23650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:50.324182 23650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:50.329252 23660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:50.329324 23657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:50.329257 23658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:50.329454 23650 server_base.cc:1034] running on GCE node
I20260806 14:58:50.329716 23650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:50.329962 23650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:50.331122 23650 hybrid_clock.cc:648] HybridClock initialized: now 1786028330331114 us; error 47 us; skew 500 ppm
Aug 06 14:58:50 dist-test-slave-68xk krb5kdc[22524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028330, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:50.333639 23650 init.cc:377] Logged in from keytab as oryx/127.11.8.66@KRBTEST.COM (short username oryx)
I20260806 14:58:50.334072 23650 webserver.cc:460] Webserver started at http://127.11.8.66:41687/ using document root <none> and password file <none>
I20260806 14:58:50.334324 23650 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:50.334411 23650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:50.335793 23650 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 14:58:50.336364 23669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:50.336525 23650 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 14:58:50.336632 23650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "3e7db4affa5f4c699e240398a8347a7c"
format_stamp: "Formatted at 2026-08-06 14:58:49 on dist-test-slave-68xk"
server_key: "c59d4804d3453a54d470511f15e4deb6"
server_key_iv: "0d6a2ab935e78d99eb3894125da0b367"
server_key_version: "encryptionkey@0"
I20260806 14:58:50.336961 23650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:50.346834 23650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:50.349115 23650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:50.349291 23650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:50.349522 23650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:50.349828 23650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:50.349882 23650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:50.349924 23650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:50.349951 23650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:50.356006 23650 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:40519
I20260806 14:58:50.356140 23833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:40519 every 8 connection(s)
I20260806 14:58:50.356441 23650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
I20260806 14:58:50.362214 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 23650
Aug 06 14:58:50 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:50.362608 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:46573
--local_ip_for_outbound_sockets=127.11.8.67
--tserver_master_addrs=127.11.8.126:45081
--webserver_port=45835
--webserver_interface=127.11.8.67
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:44007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.11.8.67
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 14:58:50 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:50.364106 23837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:50.357160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.66:59885)
0806 14:58:50.357334 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:50.357376 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:50.357860 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:50.358139 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:50.358142 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:50.358824 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:50.359012 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:50.359017 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.359685 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.359685 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:50.360464 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:50.360466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.360849 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.360850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:50.360945 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:50.361464 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:50.361473 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:50.363969 (+ 2496us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":100,"threads_started":1}
W20260806 14:58:50.364188 23649 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:50.357206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:59885 (local address 127.11.8.126:45081)
0806 14:58:50.357254 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:50.357256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:50.357467 (+ 211us) server_negotiation.cc:407] Connection header received
0806 14:58:50.357978 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:50.357979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:50.357994 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:50.358060 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:50.359113 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.359603 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.360565 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.360764 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.360833 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:50.364100 (+ 3267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.66:59885: BlockingRecv error: recv got EOF from 127.11.8.66:59885 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 14:58:50.364548 23835 heartbeater.cc:644] Failed to heartbeat to 127.11.8.126:45081 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.8.126:45081: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Aug 06 14:58:50 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:50 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:50.369802 23837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:50.364804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.66:57497)
0806 14:58:50.364859 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:50.364873 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:50.364923 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:50.365078 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:50.365079 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:50.365240 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:50.365310 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:50.365311 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.366077 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.366078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:50.367026 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:50.367030 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.367362 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.367363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:50.367447 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:50.367876 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:50.367882 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:50.369760 (+ 1878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 14:58:50.369951 23649 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:50.364868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:57497 (local address 127.11.8.126:45081)
0806 14:58:50.364927 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:50.364929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:50.364957 (+ 28us) server_negotiation.cc:407] Connection header received
0806 14:58:50.364994 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:50.364994 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:50.365002 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:50.365048 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:50.365420 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.365989 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.367106 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.367288 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.367343 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:50.369871 (+ 2528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.66:57497: BlockingRecv error: recv got EOF from 127.11.8.66:57497 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 06 14:58:50 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:50 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 14:58:50.374743 23649 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:50.370312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:33597 (local address 127.11.8.126:45081)
0806 14:58:50.370376 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:50.370378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:50.370388 (+ 10us) server_negotiation.cc:407] Connection header received
0806 14:58:50.370430 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:50.370431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:50.370441 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:50.370506 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:50.370725 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.371475 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.372114 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.372432 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.372506 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:50.374702 (+ 2196us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.66:33597: BlockingRecv error: recv got EOF from 127.11.8.66:33597 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 14:58:50.374748 23837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:50.370243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.66:33597)
0806 14:58:50.370298 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:50.370310 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:50.370352 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:50.370523 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:50.370523 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:50.370627 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:50.370651 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:50.370652 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.371566 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.371567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:50.372032 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:50.372033 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.372520 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.372521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:50.372557 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:50.372944 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:50.372949 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:50.374627 (+ 1678us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 14:58:50.374900 23835 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 14:58:50.448091 23838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:58:50.448328 23838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:58:50.448367 23838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:58:50.449505 23838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:58:50.449574 23838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:58:50.449618 23838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 14:58:50.451119 23838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:44007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/krb5kdc/oryx_127.11.8.67.keytab
--principal=oryx/127.11.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:46573
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=45835
--tserver_master_addrs=127.11.8.126:45081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:58:50.451380 23838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:58:50.451617 23838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:58:50.456627 23847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:50.456625 23849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:58:50.456636 23846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:58:50.456938 23838 server_base.cc:1034] running on GCE node
I20260806 14:58:50.457108 23838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:58:50.457321 23838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:58:50.458482 23838 hybrid_clock.cc:648] HybridClock initialized: now 1786028330458463 us; error 34 us; skew 500 ppm
Aug 06 14:58:50 dist-test-slave-68xk krb5kdc[22524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028330, etypes {rep=17 tkt=17 ses=17}, oryx/127.11.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:58:50.460877 23838 init.cc:377] Logged in from keytab as oryx/127.11.8.67@KRBTEST.COM (short username oryx)
I20260806 14:58:50.461344 23838 webserver.cc:460] Webserver started at http://127.11.8.67:45835/ using document root <none> and password file <none>
I20260806 14:58:50.461572 23838 fs_manager.cc:359] Metadata directory not provided
I20260806 14:58:50.461655 23838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:58:50.463119 23838 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 14:58:50.463745 23861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:50.463963 23838 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 14:58:50.464073 23838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "e237d7e262474e789170789129ce2cc4"
format_stamp: "Formatted at 2026-08-06 14:58:49 on dist-test-slave-68xk"
server_key: "54ecb0967708242405fd33a379266826"
server_key_iv: "58f39d4116f5e0d42049104a0e911bf7"
server_key_version: "encryptionkey@0"
I20260806 14:58:50.464419 23838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:58:50.487946 23838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:58:50.489920 23838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:58:50.490095 23838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:58:50.490370 23838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:58:50.490758 23838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:58:50.490818 23838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:50.490862 23838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:58:50.490888 23838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:58:50.496754 23838 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:46573
I20260806 14:58:50.496832 24031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:46573 every 8 connection(s)
I20260806 14:58:50.497174 23838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
I20260806 14:58:50.499215 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 23838
Aug 06 14:58:50 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:50 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 14:58:50.505411 23649 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:50.497952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:45845 (local address 127.11.8.126:45081)
0806 14:58:50.498022 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:50.498024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:50.498249 (+ 225us) server_negotiation.cc:407] Connection header received
0806 14:58:50.498895 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:50.498896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:50.498906 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:50.498988 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:50.500028 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.500533 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.501513 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.501714 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.501800 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:50.505365 (+ 3565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.67:45845: BlockingRecv error: recv got EOF from 127.11.8.67:45845 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 14:58:50.505491 24035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:50.497934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.67:45845)
0806 14:58:50.498111 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:50.498152 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:50.498799 (+ 647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:50.499041 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:50.499044 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:50.499654 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:50.499947 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:50.499955 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.500634 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.500635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:50.501419 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:50.501421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.501812 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.501813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:50.501932 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:50.502482 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:50.502491 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:50.505292 (+ 2801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":107,"threads_started":1}
W20260806 14:58:50.505729 24032 heartbeater.cc:644] Failed to heartbeat to 127.11.8.126:45081 (0 consecutive failures): Not authorized: Failed to ping master at 127.11.8.126:45081: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
I20260806 14:58:50.507501 24038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:50.499773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50222 (local address 127.11.8.126:45081)
0806 14:58:50.499866 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:50.499868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:50.499914 (+ 46us) server_negotiation.cc:407] Connection header received
0806 14:58:50.499974 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:50.499974 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:50.499983 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:50.500128 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:50.500825 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.501315 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.502126 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.502340 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.502418 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:50.503847 (+ 1429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:58:50.503851 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:58:50.503851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:58:50.503861 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:58:50.506066 (+ 2205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:50.506446 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:50.506449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:50.506450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:50.506555 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:58:50.506770 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:58:50.506772 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:58:50.506772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:58:50.507000 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:58:50.507156 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 14:58:50.507275 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 14:58:50.507384 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76,"thread_start_us":48,"threads_started":1}
Aug 06 14:58:50 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:50 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:50.510236 24035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:50.505944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.67:51973)
0806 14:58:50.506049 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:50.506060 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:50.506111 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:50.506228 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:50.506229 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:50.506363 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:50.506404 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:50.506405 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.507044 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.507045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:50.507605 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:50.507607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.508015 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.508016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:50.508111 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:50.508531 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:50.508536 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:50.510196 (+ 1660us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":88}
W20260806 14:58:50.510375 23649 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:50.506005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:51973 (local address 127.11.8.126:45081)
0806 14:58:50.506043 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:50.506045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:50.506095 (+ 50us) server_negotiation.cc:407] Connection header received
0806 14:58:50.506158 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:50.506159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:50.506168 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:50.506223 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:50.506509 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.506962 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.507695 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.507937 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.507993 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:50.510305 (+ 2312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.67:51973: BlockingRecv error: recv got EOF from 127.11.8.67:51973 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Aug 06 14:58:50 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:50 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:50.514371 24035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:50.510639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.67:52051)
0806 14:58:50.510705 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:50.510715 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:50.510758 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:50.510903 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:50.510904 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:50.511031 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:50.511054 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:50.511055 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.511672 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.511673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:50.512166 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:50.512167 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.512522 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.512523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:50.512575 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:50.512906 (+ 331us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:50.512909 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:50.514329 (+ 1420us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 14:58:50.514516 23649 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:50.510702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:52051 (local address 127.11.8.126:45081)
0806 14:58:50.510742 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:50.510744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:50.510754 (+ 10us) server_negotiation.cc:407] Connection header received
0806 14:58:50.510816 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:50.510817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:50.510825 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:50.510898 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:50.511128 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.511587 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.512251 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:50.512436 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:50.512520 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:50.514446 (+ 1926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.67:52051: BlockingRecv error: recv got EOF from 127.11.8.67:52051 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 14:58:50.514518 24032 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 14:58:51 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:51 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:51.252511 24040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:51.247763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.65:34091)
0806 14:58:51.247921 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:51.247936 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:51.248014 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:51.248195 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:51.248196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:51.248361 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:51.248412 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:51.248413 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:51.249142 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:51.249143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:51.249705 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:51.249707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:51.250120 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:51.250121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:51.250168 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:51.250678 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:51.250683 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:51.252445 (+ 1762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":79,"threads_started":1}
W20260806 14:58:51.252691 24041 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:51.247829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:34091 (local address 127.11.8.126:45081)
0806 14:58:51.248003 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:51.248006 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:51.248016 (+ 10us) server_negotiation.cc:407] Connection header received
0806 14:58:51.248096 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:51.248097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:51.248108 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:51.248171 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:51.248489 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:51.249070 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:51.249794 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:51.250013 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:51.250107 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:51.252594 (+ 2487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.65:34091: BlockingRecv error: recv got EOF from 127.11.8.65:34091 (error 108)
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":88,"threads_started":1}
Aug 06 14:58:51 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:51 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:51.380470 24042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:51.375295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.66:49361)
0806 14:58:51.375597 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:51.375609 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:51.375708 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:51.375875 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:51.375876 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:51.376077 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:51.376134 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:51.376136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:51.376840 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:51.376842 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:51.377431 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:51.377434 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:51.377813 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:51.377814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:51.377872 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:51.378370 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:51.378375 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:51.380396 (+ 2021us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":107,"threads_started":1}
W20260806 14:58:51.380592 24041 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:51.375393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:49361 (local address 127.11.8.126:45081)
0806 14:58:51.375576 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:51.375578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:51.375687 (+ 109us) server_negotiation.cc:407] Connection header received
0806 14:58:51.375774 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:51.375775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:51.375789 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:51.375874 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:51.376217 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:51.376746 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:51.377526 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:51.377735 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:51.377814 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:51.380512 (+ 2698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.66:49361: BlockingRecv error: recv got EOF from 127.11.8.66:49361 (error 108)
Metrics: {"server-negotiator.queue_time_us":146}
Aug 06 14:58:51 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:51 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:51.520308 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:51.514994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.67:60107)
0806 14:58:51.515188 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:51.515205 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:51.515294 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:51.515492 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:51.515493 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:51.515701 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:51.515758 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:51.515761 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:51.516701 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:51.516702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:51.517318 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:51.517320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:51.517743 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:51.517745 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:51.517819 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:51.518381 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:51.518386 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:51.520242 (+ 1856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":77,"threads_started":1}
W20260806 14:58:51.520438 24041 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:51.515060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:60107 (local address 127.11.8.126:45081)
0806 14:58:51.515148 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:51.515151 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:51.515276 (+ 125us) server_negotiation.cc:407] Connection header received
0806 14:58:51.515371 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:51.515372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:51.515389 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:51.515471 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:51.515848 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:51.516600 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:51.517415 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:51.517657 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:51.517745 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:51.520336 (+ 2591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.67:60107: BlockingRecv error: recv got EOF from 127.11.8.67:60107 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 14:58:52 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:52 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:52.257498 24044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:52.252939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.65:40203)
0806 14:58:52.253097 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:52.253109 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:52.253166 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:52.253328 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:52.253328 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:52.253500 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:52.253550 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:52.253551 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:52.254180 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:52.254181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:52.254806 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:52.254808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:52.255136 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:52.255137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:52.255177 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:52.255649 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:52.255654 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:52.257432 (+ 1778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":96,"threads_started":1}
W20260806 14:58:52.257634 24045 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:52.253027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:40203 (local address 127.11.8.126:45081)
0806 14:58:52.253135 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:52.253137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:52.253149 (+ 12us) server_negotiation.cc:407] Connection header received
0806 14:58:52.253241 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:52.253241 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:52.253252 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:52.253309 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:52.253613 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:52.254117 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:52.254878 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:52.255069 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:52.255126 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:52.257525 (+ 2399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.65:40203: BlockingRecv error: recv got EOF from 127.11.8.65:40203 (error 108)
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":51,"threads_started":1}
Aug 06 14:58:52 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:52 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:52.385869 24046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:52.381057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.66:43541)
0806 14:58:52.381246 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:52.381259 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:52.381327 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:52.381491 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:52.381493 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:52.381663 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:52.381720 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:52.381721 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:52.382537 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:52.382539 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:52.383066 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:52.383068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:52.383446 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:52.383447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:52.383514 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:52.383954 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:52.383959 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:52.385803 (+ 1844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":78,"threads_started":1}
W20260806 14:58:52.385987 24045 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:52.381257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:43541 (local address 127.11.8.126:45081)
0806 14:58:52.381333 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:52.381334 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:52.381345 (+ 11us) server_negotiation.cc:407] Connection header received
0806 14:58:52.381371 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:52.381371 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:52.381382 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:52.381442 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:52.381798 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:52.382449 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:52.383158 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:52.383358 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:52.383435 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:52.385894 (+ 2459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.66:43541: BlockingRecv error: recv got EOF from 127.11.8.66:43541 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 06 14:58:52 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:52 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:52.525923 24048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:52.520808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.67:39927)
0806 14:58:52.520968 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:52.520980 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:52.521040 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:52.521184 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:52.521185 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:52.521360 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:52.521409 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:52.521411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:52.522387 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:52.522388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:52.523059 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:52.523061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:52.523425 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:52.523426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:52.523467 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:52.523971 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:52.523976 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:52.525860 (+ 1884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":113,"threads_started":1}
W20260806 14:58:52.526095 24045 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:52.520950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:39927 (local address 127.11.8.126:45081)
0806 14:58:52.521010 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:52.521012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:52.521035 (+ 23us) server_negotiation.cc:407] Connection header received
0806 14:58:52.521090 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:52.521091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:52.521106 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:52.521161 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:52.521483 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:52.522301 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:52.523150 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:52.523352 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:52.523416 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:52.526018 (+ 2602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.67:39927: BlockingRecv error: recv got EOF from 127.11.8.67:39927 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 06 14:58:53 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:53 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:53.263098 24049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:53.258047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.65:54661)
0806 14:58:53.258259 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:53.258271 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:53.258337 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:53.258530 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:53.258531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:53.258778 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:53.258831 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:53.258833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:53.259599 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:53.259600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:53.260136 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:53.260138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:53.260506 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:53.260507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:53.260546 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:53.261062 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:53.261068 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:53.263022 (+ 1954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":137,"threads_started":1}
W20260806 14:58:53.263264 24050 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:53.258167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:54661 (local address 127.11.8.126:45081)
0806 14:58:53.258290 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:53.258292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:53.258337 (+ 45us) server_negotiation.cc:407] Connection header received
0806 14:58:53.258381 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:53.258381 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:53.258391 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:53.258510 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:53.258930 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:53.259521 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:53.260229 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:53.260430 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:53.260497 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:53.263155 (+ 2658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.65:54661: BlockingRecv error: recv got EOF from 127.11.8.65:54661 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":55,"threads_started":1}
Aug 06 14:58:53 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:53 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:53.391124 24051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:53.386332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.66:46617)
0806 14:58:53.386481 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:53.386492 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:53.386564 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:53.386702 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:53.386703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:53.386888 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:53.386937 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:53.386939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:53.387612 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:53.387615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:53.388170 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:53.388172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:53.388552 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:53.388553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:53.388621 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:53.389129 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:53.389138 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:53.391046 (+ 1908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
W20260806 14:58:53.391269 24050 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:53.386488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:46617 (local address 127.11.8.126:45081)
0806 14:58:53.386562 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:53.386564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:53.386574 (+ 10us) server_negotiation.cc:407] Connection header received
0806 14:58:53.386626 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:53.386627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:53.386639 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:53.386686 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:53.387026 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:53.387527 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:53.388269 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:53.388472 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:53.388552 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:53.391186 (+ 2634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.66:46617: BlockingRecv error: recv got EOF from 127.11.8.66:46617 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 06 14:58:53 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:53 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:53.531044 24052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:53.526465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.67:60907)
0806 14:58:53.526637 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:53.526649 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:53.526721 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:53.526860 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:53.526861 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:53.527037 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:53.527093 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:53.527096 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:53.527726 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:53.527727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:53.528276 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:53.528278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:53.528614 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:53.528615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:53.528657 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:53.529188 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:53.529195 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:53.530977 (+ 1782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":90,"threads_started":1}
W20260806 14:58:53.531162 24050 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:53.526488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:60907 (local address 127.11.8.126:45081)
0806 14:58:53.526536 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:53.526538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:53.526725 (+ 187us) server_negotiation.cc:407] Connection header received
0806 14:58:53.526782 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:53.526782 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:53.526795 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:53.526837 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:53.527161 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:53.527645 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:53.528350 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:53.528531 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:53.528584 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:53.531083 (+ 2499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.67:60907: BlockingRecv error: recv got EOF from 127.11.8.67:60907 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 06 14:58:54 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:54 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:54.274083 24054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:54.263614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.65:42649)
0806 14:58:54.263799 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:54.263811 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:54.263866 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:54.264017 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:54.264018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:54.264183 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:54.264238 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:54.264239 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:54.264930 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:54.264931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:54.265532 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:54.265534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:54.265887 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:54.265888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:54.265986 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:54.271709 (+ 5723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:54.271714 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:54.273952 (+ 2238us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":121,"threads_started":1}
W20260806 14:58:54.274206 24055 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:54.263715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:42649 (local address 127.11.8.126:45081)
0806 14:58:54.263876 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:54.263878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:54.263908 (+ 30us) server_negotiation.cc:407] Connection header received
0806 14:58:54.263933 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:54.263933 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:54.263944 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:54.264003 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:54.264339 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:54.264857 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:54.265617 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:54.265827 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:54.265913 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:54.274069 (+ 8156us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.65:42649: BlockingRecv error: recv got EOF from 127.11.8.65:42649 (error 108)
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":89,"threads_started":1}
Aug 06 14:58:54 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:54 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:54.396725 24060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:54.391663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.66:60479)
0806 14:58:54.391841 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:54.391852 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:54.391931 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:54.392065 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:54.392065 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:54.392252 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:54.392301 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:54.392303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:54.393004 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:54.393006 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:54.393564 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:54.393566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:54.393939 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:54.393940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:54.393998 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:54.394618 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:54.394623 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:54.396612 (+ 1989us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":79,"threads_started":1}
W20260806 14:58:54.396765 24055 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:54.391707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:60479 (local address 127.11.8.126:45081)
0806 14:58:54.391754 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:54.391757 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:54.391910 (+ 153us) server_negotiation.cc:407] Connection header received
0806 14:58:54.391986 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:54.391987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:54.392001 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:54.392055 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:54.392390 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:54.392910 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:54.393648 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:54.393861 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:54.393941 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:54.396703 (+ 2762us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.66:60479: BlockingRecv error: recv got EOF from 127.11.8.66:60479 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 06 14:58:54 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:54 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:54.536715 24061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:54.531593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.67:37589)
0806 14:58:54.531729 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:54.531742 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:54.531816 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:54.531949 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:54.531950 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:54.532114 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:54.532165 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:54.532167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:54.532920 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:54.532922 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:54.533598 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:54.533601 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:54.533983 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:54.533985 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:54.534048 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:54.534619 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:54.534624 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:54.536621 (+ 1997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
W20260806 14:58:54.536784 24055 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:54.531709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:37589 (local address 127.11.8.126:45081)
0806 14:58:54.531759 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:54.531761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:54.531809 (+ 48us) server_negotiation.cc:407] Connection header received
0806 14:58:54.531863 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:54.531864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:54.531876 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:54.531933 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:54.532246 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:54.532828 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:54.533692 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:54.533900 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:54.533982 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:54.536719 (+ 2737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.67:37589: BlockingRecv error: recv got EOF from 127.11.8.67:37589 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 06 14:58:55 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:55 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:55.279120 24064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:55.274668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.65:51409)
0806 14:58:55.274814 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:55.274825 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:55.274873 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:55.275019 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:55.275020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:55.275186 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:55.275241 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:55.275242 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:55.275895 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:55.275896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:55.276422 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:55.276424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:55.276758 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:55.276759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:55.276799 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:55.277260 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:55.277265 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:55.279061 (+ 1796us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
W20260806 14:58:55.279240 24065 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:55.274735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:51409 (local address 127.11.8.126:45081)
0806 14:58:55.274877 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:55.274879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:55.274912 (+ 33us) server_negotiation.cc:407] Connection header received
0806 14:58:55.274934 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:55.274935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:55.274946 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:55.274998 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:55.275314 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:55.275826 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:55.276500 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:55.276691 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:55.276744 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:55.279157 (+ 2413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.65:51409: BlockingRecv error: recv got EOF from 127.11.8.65:51409 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
Aug 06 14:58:55 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:55 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:55.402549 24067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:55.397261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.66:53275)
0806 14:58:55.397423 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:55.397435 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:55.397493 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:55.397656 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:55.397657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:55.397845 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:55.397895 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:55.397897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:55.398725 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:55.398727 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:55.399463 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:55.399466 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:55.399902 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:55.399904 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:55.399971 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:55.400602 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:55.400607 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:55.402490 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":63,"threads_started":1}
W20260806 14:58:55.402667 24065 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:55.397430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:53275 (local address 127.11.8.126:45081)
0806 14:58:55.397500 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:55.397507 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:55.397516 (+ 9us) server_negotiation.cc:407] Connection header received
0806 14:58:55.397546 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:55.397546 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:55.397556 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:55.397642 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:55.398079 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:55.398615 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:55.399546 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:55.399831 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:55.399903 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:55.402574 (+ 2671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.66:53275: BlockingRecv error: recv got EOF from 127.11.8.66:53275 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 06 14:58:55 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:55 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:55.542649 24068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:55.537280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.67:47799)
0806 14:58:55.537455 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:55.537468 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:55.537543 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:55.537738 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:55.537739 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:55.537913 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:55.537964 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:55.537965 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:55.538654 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:55.538656 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:55.539252 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:55.539255 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:55.539685 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:55.539686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:55.539748 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:55.540273 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:55.540279 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:55.542585 (+ 2306us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":119,"threads_started":1}
W20260806 14:58:55.542797 24065 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:55.537366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:47799 (local address 127.11.8.126:45081)
0806 14:58:55.537456 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:55.537460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:55.537533 (+ 73us) server_negotiation.cc:407] Connection header received
0806 14:58:55.537635 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:55.537636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:55.537652 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:55.537718 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:55.538030 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:55.538555 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:55.539371 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:55.539596 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:55.539681 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:55.542683 (+ 3002us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.67:47799: BlockingRecv error: recv got EOF from 127.11.8.67:47799 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 14:58:56 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:56 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:56.285006 24069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:56.279670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.65:43247)
0806 14:58:56.279839 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:56.279851 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:56.279906 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:56.280076 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:56.280077 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:56.280259 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:56.280311 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:56.280313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:56.280995 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:56.280996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:56.281579 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:56.281582 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:56.282000 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:56.282002 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:56.282058 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:56.282675 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:56.282680 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:56.284945 (+ 2265us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":101,"threads_started":1}
W20260806 14:58:56.285195 24070 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:56.279798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:43247 (local address 127.11.8.126:45081)
0806 14:58:56.279930 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:56.279932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:56.279941 (+ 9us) server_negotiation.cc:407] Connection header received
0806 14:58:56.279975 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:56.279976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:56.279986 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:56.280054 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:56.280381 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:56.280900 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:56.281698 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:56.281905 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:56.281972 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:56.285055 (+ 3083us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.65:43247: BlockingRecv error: recv got EOF from 127.11.8.65:43247 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":81,"threads_started":1}
Aug 06 14:58:56 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:56 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:56.407745 24072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:56.403011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.66:41567)
0806 14:58:56.403154 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:56.403164 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:56.403232 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:56.403374 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:56.403375 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:56.403547 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:56.403596 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:56.403597 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:56.404230 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:56.404231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:56.404871 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:56.404873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:56.405263 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:56.405264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:56.405313 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:56.405770 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:56.405775 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:56.407680 (+ 1905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":84,"threads_started":1}
W20260806 14:58:56.407913 24070 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:56.403100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:41567 (local address 127.11.8.126:45081)
0806 14:58:56.403173 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:56.403175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:56.403233 (+ 58us) server_negotiation.cc:407] Connection header received
0806 14:58:56.403296 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:56.403297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:56.403309 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:56.403353 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:56.403662 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:56.404161 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:56.404933 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:56.405168 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:56.405246 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:56.407821 (+ 2575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.66:41567: BlockingRecv error: recv got EOF from 127.11.8.66:41567 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 06 14:58:56 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:56 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:56.548272 24073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:56.543206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.67:52385)
0806 14:58:56.543389 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:56.543404 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:56.543466 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:56.543679 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:56.543680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:56.543857 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:56.543909 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:56.543910 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:56.544631 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:56.544632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:56.545186 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:56.545189 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:56.545576 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:56.545577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:56.545630 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:56.546205 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:56.546211 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:56.548211 (+ 2000us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":134,"threads_started":1}
W20260806 14:58:56.548450 24070 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:56.543317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:52385 (local address 127.11.8.126:45081)
0806 14:58:56.543403 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:56.543406 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:56.543462 (+ 56us) server_negotiation.cc:407] Connection header received
0806 14:58:56.543528 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:56.543536 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:56.543549 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:56.543618 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:56.544011 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:56.544526 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:56.545266 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:56.545459 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:56.545524 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:56.548366 (+ 2842us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.67:52385: BlockingRecv error: recv got EOF from 127.11.8.67:52385 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 06 14:58:57 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:57 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:57.290318 24074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:57.285638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.65:56891)
0806 14:58:57.285813 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:57.285831 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:57.285889 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:57.286010 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:57.286011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:57.286230 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:57.286283 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:57.286285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:57.286947 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:57.286948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:57.287563 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:57.287565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:57.287902 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:57.287904 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:57.287953 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:57.288413 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:57.288418 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:57.290255 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":109,"threads_started":1}
W20260806 14:58:57.290449 24075 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:57.285666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:56891 (local address 127.11.8.126:45081)
0806 14:58:57.285838 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:57.285839 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:57.285875 (+ 36us) server_negotiation.cc:407] Connection header received
0806 14:58:57.285934 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:57.285935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:57.285945 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:57.286004 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:57.286358 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:57.286874 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:57.287643 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:57.287833 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:57.287884 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:57.290352 (+ 2468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.65:56891: BlockingRecv error: recv got EOF from 127.11.8.65:56891 (error 108)
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":115,"threads_started":1}
Aug 06 14:58:57 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:57 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:57.412998 24077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:57.408297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.66:34321)
0806 14:58:57.408499 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:57.408511 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:57.408600 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:57.408754 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:57.408755 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:57.408930 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:57.408981 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:57.408982 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:57.409660 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:57.409661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:57.410206 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:57.410208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:57.410544 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:57.410545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:57.410607 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:57.411115 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:57.411120 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:57.412938 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":130,"threads_started":1}
W20260806 14:58:57.413102 24075 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:57.408365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:34321 (local address 127.11.8.126:45081)
0806 14:58:57.408445 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:57.408448 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:57.408624 (+ 176us) server_negotiation.cc:407] Connection header received
0806 14:58:57.408658 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:57.408658 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:57.408672 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:57.408735 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:57.409058 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:57.409568 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:57.410288 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:57.410487 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:57.410537 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:57.413031 (+ 2494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.66:34321: BlockingRecv error: recv got EOF from 127.11.8.66:34321 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 06 14:58:57 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:57 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:57.554293 24078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:57.548765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.67:48563)
0806 14:58:57.548959 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:57.548972 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:57.549074 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:57.549270 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:57.549271 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:57.549476 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:57.549526 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:57.549528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:57.550356 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:57.550357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:57.551136 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:57.551138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:57.551595 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:57.551596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:57.551663 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:57.552177 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:57.552182 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:57.554230 (+ 2048us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":101,"threads_started":1}
W20260806 14:58:57.554471 24075 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:57.548857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:48563 (local address 127.11.8.126:45081)
0806 14:58:57.548939 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:57.548941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:57.549052 (+ 111us) server_negotiation.cc:407] Connection header received
0806 14:58:57.549148 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:57.549149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:57.549173 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:57.549247 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:57.549623 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:57.550247 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:57.551239 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:57.551471 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:57.551589 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:57.554359 (+ 2770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.67:48563: BlockingRecv error: recv got EOF from 127.11.8.67:48563 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 14:58:58 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:58 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:58.296087 24079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:58.290992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.65:41543)
0806 14:58:58.291191 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:58.291206 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:58.291274 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:58.291435 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:58.291436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:58.291607 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:58.291658 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:58.291660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:58.292309 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:58.292310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:58.292831 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:58.292832 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:58.293315 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:58.293317 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:58.293401 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:58.293938 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:58.293944 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:58.296020 (+ 2076us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":101,"threads_started":1}
W20260806 14:58:58.296217 24080 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:58.291153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:41543 (local address 127.11.8.126:45081)
0806 14:58:58.291311 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:58.291313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:58.291321 (+ 8us) server_negotiation.cc:407] Connection header received
0806 14:58:58.291345 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:58.291345 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:58.291356 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:58.291416 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:58.291729 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:58.292243 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:58.292917 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:58.293231 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:58.293307 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:58.296127 (+ 2820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.65:41543: BlockingRecv error: recv got EOF from 127.11.8.65:41543 (error 108)
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":88,"threads_started":1}
Aug 06 14:58:58 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:58 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:58.418783 24081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:58.413666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.66:44529)
0806 14:58:58.413848 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:58.413861 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:58.413930 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:58.414064 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:58.414065 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:58.414252 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:58.414304 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:58.414306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:58.415197 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:58.415199 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:58.415783 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:58.415785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:58.416123 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:58.416124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:58.416173 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:58.416747 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:58.416753 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:58.418704 (+ 1951us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":119,"threads_started":1}
W20260806 14:58:58.418886 24080 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:58.413735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:44529 (local address 127.11.8.126:45081)
0806 14:58:58.413809 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:58.413812 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:58.413930 (+ 118us) server_negotiation.cc:407] Connection header received
0806 14:58:58.413979 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:58.413980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:58.413992 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:58.414041 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:58.414375 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:58.415114 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:58.415865 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:58.416046 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:58.416097 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:58.418819 (+ 2722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.66:44529: BlockingRecv error: recv got EOF from 127.11.8.66:44529 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 14:58:58 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:58 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:58.559334 24082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:58.554804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.67:50675)
0806 14:58:58.554923 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:58.554941 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:58.555001 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:58.555111 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:58.555112 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:58.555287 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:58.555336 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:58.555338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:58.555976 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:58.555977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:58.556513 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:58.556514 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:58.556847 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:58.556847 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:58.556886 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:58.557416 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:58.557421 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:58.559276 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":59,"threads_started":1}
W20260806 14:58:58.559537 24080 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:58.554883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:50675 (local address 127.11.8.126:45081)
0806 14:58:58.554939 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:58.554941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:58.554986 (+ 45us) server_negotiation.cc:407] Connection header received
0806 14:58:58.555052 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:58.555053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:58.555065 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:58.555106 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:58.555404 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:58.555907 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:58.556589 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:58.556782 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:58.556835 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:58.559449 (+ 2614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.67:50675: BlockingRecv error: recv got EOF from 127.11.8.67:50675 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 06 14:58:59 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:59 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:59.304636 24084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:59.296583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.65:59459)
0806 14:58:59.296739 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:59.296760 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:59.296822 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:59.297012 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:59.297013 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:59.297253 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:59.297305 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:59.297307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:59.298065 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:59.298066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:59.298690 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:59.298692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:59.299050 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:59.299051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:59.299105 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:59.302658 (+ 3553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:59.302664 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:59.304578 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":77,"threads_started":1}
W20260806 14:58:59.304816 24086 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:59.296629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:59459 (local address 127.11.8.126:45081)
0806 14:58:59.296780 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:59.296782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:59.296805 (+ 23us) server_negotiation.cc:407] Connection header received
0806 14:58:59.296911 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:59.296913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:59.296925 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:59.296999 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:59.297409 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:59.297988 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:59.298769 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:59.298971 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:59.299036 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:59.304741 (+ 5705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.65:59459: BlockingRecv error: recv got EOF from 127.11.8.65:59459 (error 108)
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
Aug 06 14:58:59 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:59 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:59.424649 24087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:59.419349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.66:59829)
0806 14:58:59.419575 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:59.419586 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:59.419638 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:59.419859 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:59.419861 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:59.420039 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:59.420092 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:59.420094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:59.420826 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:59.420827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:59.421364 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:59.421366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:59.421906 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:59.421908 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:59.421999 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:59.422586 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:59.422591 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:59.424591 (+ 2000us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":105,"threads_started":1}
W20260806 14:58:59.424724 24086 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:59.419500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:59829 (local address 127.11.8.126:45081)
0806 14:58:59.419605 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:59.419607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:59.419622 (+ 15us) server_negotiation.cc:407] Connection header received
0806 14:58:59.419708 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:59.419709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:59.419722 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:59.419785 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:59.420180 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:59.420744 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:59.421436 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:59.421820 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:59.421893 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:59.424673 (+ 2780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.66:59829: BlockingRecv error: recv got EOF from 127.11.8.66:59829 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 06 14:58:59 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:58:59 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:58:59.565425 24090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:58:59.559887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.67:57273)
0806 14:58:59.560065 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 14:58:59.560082 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 14:58:59.560149 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:58:59.560291 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:58:59.560291 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:58:59.560463 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:58:59.560514 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:59.560516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:59.561229 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:59.561230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:59.561771 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:58:59.561772 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:58:59.562180 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:58:59.562181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:58:59.562240 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:59.562902 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:58:59.562908 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:58:59.565326 (+ 2418us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":131,"threads_started":1}
W20260806 14:58:59.565467 24086 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:58:59.559962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:57273 (local address 127.11.8.126:45081)
0806 14:58:59.560037 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 14:58:59.560040 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:58:59.560150 (+ 110us) server_negotiation.cc:407] Connection header received
0806 14:58:59.560209 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:58:59.560210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:58:59.560222 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:58:59.560264 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:58:59.560634 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:59.561141 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:59.561862 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:58:59.562055 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:58:59.562126 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:58:59.565404 (+ 3278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.67:57273: BlockingRecv error: recv got EOF from 127.11.8.67:57273 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 06 14:59:00 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:59:00 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:59:00.310043 24091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:00.305202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.65:41139)
0806 14:59:00.305369 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:00.305383 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:00.305442 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:00.305628 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:00.305629 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:00.305890 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:00.305950 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:00.305953 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:00.306666 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:00.306667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:00.307204 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:00.307206 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:00.307561 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:00.307562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:00.307605 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:00.308132 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:59:00.308137 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:00.309972 (+ 1835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":88,"threads_started":1}
W20260806 14:59:00.310153 24092 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:59:00.305320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:41139 (local address 127.11.8.126:45081)
0806 14:59:00.305450 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:00.305468 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:00.305477 (+ 9us) server_negotiation.cc:407] Connection header received
0806 14:59:00.305505 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:00.305506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:00.305520 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:00.305621 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:00.306065 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:00.306590 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:00.307290 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:00.307490 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:00.307554 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:00.310075 (+ 2521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.65:41139: BlockingRecv error: recv got EOF from 127.11.8.65:41139 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":52,"threads_started":1}
Aug 06 14:59:00 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:59:00 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:59:00.430251 24094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:00.425279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.66:60539)
0806 14:59:00.425457 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:00.425469 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:00.425538 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:00.425746 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:00.425748 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:00.425954 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:00.426005 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:00.426007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:00.426830 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:00.426831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:00.427362 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:00.427363 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:00.427768 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:00.427769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:00.427810 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:00.428333 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:59:00.428338 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:00.430176 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":95,"threads_started":1}
W20260806 14:59:00.430367 24092 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:59:00.425334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:60539 (local address 127.11.8.126:45081)
0806 14:59:00.425389 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:00.425391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:00.425530 (+ 139us) server_negotiation.cc:407] Connection header received
0806 14:59:00.425620 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:00.425621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:00.425637 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:00.425716 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:00.426078 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:00.426732 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:00.427469 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:00.427695 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:00.427755 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:00.430272 (+ 2517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.66:60539: BlockingRecv error: recv got EOF from 127.11.8.66:60539 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 06 14:59:00 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:59:00 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:59:00.570930 24095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:00.565912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.67:44737)
0806 14:59:00.566053 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:00.566063 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:00.566146 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:00.566293 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:00.566294 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:00.566543 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:00.566592 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:00.566594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:00.567601 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:00.567603 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:00.568153 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:00.568155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:00.568563 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:00.568564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:00.568626 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:00.569056 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:59:00.569061 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:00.570866 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
W20260806 14:59:00.571022 24092 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:59:00.566047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:44737 (local address 127.11.8.126:45081)
0806 14:59:00.566125 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:00.566127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:00.566138 (+ 11us) server_negotiation.cc:407] Connection header received
0806 14:59:00.566213 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:00.566214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:00.566231 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:00.566278 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:00.566670 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:00.567525 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:00.568264 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:00.568470 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:00.568554 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:00.570955 (+ 2401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.67:44737: BlockingRecv error: recv got EOF from 127.11.8.67:44737 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 14:59:01 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:59:01 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:59:01.315442 24096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:01.310670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.65:35687)
0806 14:59:01.310833 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:01.310845 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:01.310904 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:01.311023 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:01.311024 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:01.311210 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:01.311260 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:01.311262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:01.311949 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:01.311950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:01.312560 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:01.312562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:01.312941 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:01.312942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:01.312994 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:01.313475 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:59:01.313481 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:01.315368 (+ 1887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":112,"threads_started":1}
W20260806 14:59:01.315518 24097 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:59:01.310777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:35687 (local address 127.11.8.126:45081)
0806 14:59:01.310888 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:01.310891 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:01.310901 (+ 10us) server_negotiation.cc:407] Connection header received
0806 14:59:01.310953 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:01.310953 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:01.310963 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:01.311020 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:01.311333 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:01.311851 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:01.312657 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:01.312865 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:01.312918 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:01.315463 (+ 2545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.65:35687: BlockingRecv error: recv got EOF from 127.11.8.65:35687 (error 108)
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":54,"threads_started":1}
Aug 06 14:59:01 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:59:01 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:59:01.435899 24098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:01.430739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.66:33753)
0806 14:59:01.430962 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:01.430982 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:01.431078 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:01.431256 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:01.431257 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:01.431474 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:01.431522 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:01.431524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:01.432309 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:01.432310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:01.432903 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:01.432906 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:01.433343 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:01.433344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:01.433419 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:01.433876 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:59:01.433880 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:01.435830 (+ 1950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":131,"threads_started":1}
W20260806 14:59:01.436053 24097 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:59:01.430807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:33753 (local address 127.11.8.126:45081)
0806 14:59:01.430917 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:01.430920 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:01.431056 (+ 136us) server_negotiation.cc:407] Connection header received
0806 14:59:01.431182 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:01.431182 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:01.431196 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:01.431250 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:01.431622 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:01.432200 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:01.433003 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:01.433241 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:01.433340 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:01.435960 (+ 2620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.66:33753: BlockingRecv error: recv got EOF from 127.11.8.66:33753 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 06 14:59:01 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:59:01 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:59:01.576232 24099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:01.571460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.67:56351)
0806 14:59:01.571631 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:01.571641 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:01.571705 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:01.571848 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:01.571849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:01.572026 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:01.572073 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:01.572074 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:01.572747 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:01.572747 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:01.573283 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:01.573285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:01.573660 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:01.573660 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:01.573707 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:01.574225 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:59:01.574231 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:01.576157 (+ 1926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":110,"threads_started":1}
W20260806 14:59:01.576361 24097 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:59:01.571536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:56351 (local address 127.11.8.126:45081)
0806 14:59:01.571592 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:01.571595 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:01.571686 (+ 91us) server_negotiation.cc:407] Connection header received
0806 14:59:01.571764 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:01.571765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:01.571778 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:01.571824 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:01.572142 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:01.572677 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:01.573376 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:01.573585 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:01.573652 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:01.576257 (+ 2605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.67:56351: BlockingRecv error: recv got EOF from 127.11.8.67:56351 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 06 14:59:02 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:59:02 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:59:02.320811 24100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:02.316020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.65:48771)
0806 14:59:02.316195 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:02.316211 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:02.316289 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:02.316426 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:02.316427 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:02.316607 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:02.316656 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:02.316658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:02.317329 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:02.317329 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:02.317866 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:02.317868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:02.318237 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:02.318238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:02.318277 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:02.318778 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:59:02.318783 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:02.320741 (+ 1958us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":80,"threads_started":1}
W20260806 14:59:02.320911 24101 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:59:02.316121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:48771 (local address 127.11.8.126:45081)
0806 14:59:02.316295 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:02.316297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:02.316308 (+ 11us) server_negotiation.cc:407] Connection header received
0806 14:59:02.316332 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:02.316332 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:02.316344 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:02.316402 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:02.316744 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:02.317260 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:02.317947 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:02.318169 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:02.318225 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:02.320836 (+ 2611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.65:48771: BlockingRecv error: recv got EOF from 127.11.8.65:48771 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":107,"threads_started":1}
Aug 06 14:59:02 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:59:02 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:59:02.441264 24102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:02.436370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.66:38231)
0806 14:59:02.436545 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:02.436558 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:02.436642 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:02.436801 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:02.436802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:02.436968 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:02.437016 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:02.437017 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:02.437663 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:02.437664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:02.438229 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:02.438231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:02.438596 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:02.438597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:02.438644 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:02.439157 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:59:02.439162 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:02.441118 (+ 1956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":92,"threads_started":1}
W20260806 14:59:02.441255 24101 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:59:02.436480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:38231 (local address 127.11.8.126:45081)
0806 14:59:02.436537 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:02.436539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:02.436620 (+ 81us) server_negotiation.cc:407] Connection header received
0806 14:59:02.436715 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:02.436716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:02.436727 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:02.436776 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:02.437113 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:02.437597 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:02.438317 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:02.438528 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:02.438584 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:02.441203 (+ 2619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.66:38231: BlockingRecv error: recv got EOF from 127.11.8.66:38231 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 06 14:59:02 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:59:02 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:59:02.581902 24103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:02.576766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.67:39461)
0806 14:59:02.576901 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:02.576917 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:02.576975 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:02.577158 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:02.577159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:02.577339 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:02.577387 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:02.577388 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:02.578024 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:02.578025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:02.578710 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:02.578713 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:02.579062 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:02.579063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:02.579144 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:02.579798 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:59:02.579805 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:02.581828 (+ 2023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":77,"threads_started":1}
W20260806 14:59:02.582053 24101 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:59:02.576854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:39461 (local address 127.11.8.126:45081)
0806 14:59:02.577030 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:02.577033 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:02.577050 (+ 17us) server_negotiation.cc:407] Connection header received
0806 14:59:02.577074 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:02.577075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:02.577087 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:02.577144 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:02.577455 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:02.577950 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:02.578787 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:02.578984 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:02.579038 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:02.581980 (+ 2942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.67:39461: BlockingRecv error: recv got EOF from 127.11.8.67:39461 (error 108)
Metrics: {"server-negotiator.queue_time_us":147}
Aug 06 14:59:03 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:59:03 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:59:03.326138 24107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:03.321357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.65:59345)
0806 14:59:03.321525 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:03.321537 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:03.321589 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:03.321728 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:03.321729 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:03.321915 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:03.321964 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:03.321966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:03.322641 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:03.322643 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:03.323243 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:03.323245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:03.323605 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:03.323606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:03.323659 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:03.324145 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:59:03.324150 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:03.326034 (+ 1884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":98,"threads_started":1}
W20260806 14:59:03.326210 24108 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:59:03.321435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:59345 (local address 127.11.8.126:45081)
0806 14:59:03.321579 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:03.321580 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:03.321589 (+ 9us) server_negotiation.cc:407] Connection header received
0806 14:59:03.321646 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:03.321647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:03.321657 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:03.321717 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:03.322039 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:03.322565 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:03.323323 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:03.323515 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:03.323570 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:03.326123 (+ 2553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.65:59345: BlockingRecv error: recv got EOF from 127.11.8.65:59345 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
Aug 06 14:59:03 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:59:03 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:59:03.446399 24110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:03.441758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.66:44065)
0806 14:59:03.441909 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:03.441920 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:03.441969 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:03.442102 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:03.442103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:03.442276 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:03.442325 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:03.442326 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:03.443004 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:03.443005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:03.443596 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:03.443598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:03.443934 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:03.443936 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:03.443979 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:03.444469 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:59:03.444474 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:03.446339 (+ 1865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
W20260806 14:59:03.446547 24108 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:59:03.441897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:44065 (local address 127.11.8.126:45081)
0806 14:59:03.441944 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:03.441948 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:03.441962 (+ 14us) server_negotiation.cc:407] Connection header received
0806 14:59:03.442030 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:03.442030 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:03.442040 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:03.442081 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:03.442403 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:03.442931 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:03.443674 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:03.443868 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:03.443919 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:03.446476 (+ 2557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.66:44065: BlockingRecv error: recv got EOF from 127.11.8.66:44065 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 06 14:59:03 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:59:03 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:59:03.587466 24111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:03.582437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.67:35521)
0806 14:59:03.582638 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:03.582651 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:03.582733 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:03.582876 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:03.582877 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:03.583070 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:03.583117 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:03.583119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:03.583916 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:03.583918 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:03.584489 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:03.584491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:03.584881 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:03.584883 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:03.584945 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:03.585503 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:59:03.585516 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:03.587356 (+ 1840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":121,"threads_started":1}
W20260806 14:59:03.587503 24108 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:59:03.582548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:35521 (local address 127.11.8.126:45081)
0806 14:59:03.582623 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:03.582626 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:03.582722 (+ 96us) server_negotiation.cc:407] Connection header received
0806 14:59:03.582798 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:03.582799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:03.582814 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:03.582861 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:03.583222 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:03.583823 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:03.584578 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:03.584794 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:03.584869 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:03.587437 (+ 2568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.67:35521: BlockingRecv error: recv got EOF from 127.11.8.67:35521 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 06 14:59:04 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:59:04 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:59:04.339566 24112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:04.326709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.65:58057)
0806 14:59:04.326874 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:04.326886 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:04.326946 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:04.327096 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:04.327097 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:04.327302 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:04.327354 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:04.327356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:04.328039 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:04.328041 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:04.328918 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:04.328922 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:04.329307 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:04.329308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:04.329349 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:04.337359 (+ 8010us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:59:04.337365 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:04.339424 (+ 2059us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":104,"threads_started":1}
W20260806 14:59:04.339614 24113 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:59:04.326796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:58057 (local address 127.11.8.126:45081)
0806 14:59:04.326942 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:04.326944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:04.326955 (+ 11us) server_negotiation.cc:407] Connection header received
0806 14:59:04.326997 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:04.326998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:04.327008 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:04.327076 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:04.327432 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:04.327953 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:04.329010 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:04.329216 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:04.329277 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:04.339535 (+ 10258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.65:58057: BlockingRecv error: recv got EOF from 127.11.8.65:58057 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":86,"threads_started":1}
Aug 06 14:59:04 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:59:04 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:59:04.451916 24114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:04.447014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.66:42245)
0806 14:59:04.447193 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:04.447206 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:04.447292 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:04.447460 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:04.447462 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:04.447665 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:04.447730 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:04.447732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:04.448428 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:04.448430 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:04.448972 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:04.448974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:04.449362 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:04.449363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:04.449425 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:04.449907 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:59:04.449912 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:04.451841 (+ 1929us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":73,"threads_started":1}
W20260806 14:59:04.452055 24113 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:59:04.447079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:42245 (local address 127.11.8.126:45081)
0806 14:59:04.447218 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:04.447220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:04.447278 (+ 58us) server_negotiation.cc:407] Connection header received
0806 14:59:04.447352 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:04.447353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:04.447368 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:04.447436 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:04.447817 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:04.448328 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:04.449064 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:04.449276 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:04.449359 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:04.451971 (+ 2612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.66:42245: BlockingRecv error: recv got EOF from 127.11.8.66:42245 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 14:59:04 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:59:04 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.67@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:59:04.592720 24115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:04.588072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.67:39269)
0806 14:59:04.588217 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:04.588229 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:04.588303 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:04.588434 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:04.588435 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:04.588611 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:04.588660 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:04.588661 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:04.589337 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:04.589338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:04.589866 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:04.589867 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:04.590254 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:04.590255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:04.590296 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:04.590875 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:59:04.590880 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:04.592626 (+ 1746us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
W20260806 14:59:04.592823 24113 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:59:04.588140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:39269 (local address 127.11.8.126:45081)
0806 14:59:04.588245 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:04.588247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:04.588311 (+ 64us) server_negotiation.cc:407] Connection header received
0806 14:59:04.588343 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:04.588343 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:04.588357 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:04.588416 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:04.588729 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:04.589269 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:04.589951 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:04.590184 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:04.590246 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:04.592724 (+ 2478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.67:39269: BlockingRecv error: recv got EOF from 127.11.8.67:39269 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 06 14:59:05 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:59:05 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.65@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:59:05.345606 24116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:05.340280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.65:57443)
0806 14:59:05.340432 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:05.340442 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:05.340496 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:05.340687 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:05.340688 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:05.340857 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:05.340909 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:05.340911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:05.341765 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:05.341767 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:05.342360 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:05.342363 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:05.342862 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:05.342864 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:05.342937 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:05.343457 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:59:05.343462 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:05.345533 (+ 2071us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":101,"threads_started":1}
W20260806 14:59:05.345685 24117 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:59:05.340376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:57443 (local address 127.11.8.126:45081)
0806 14:59:05.340519 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:05.340546 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:05.340555 (+ 9us) server_negotiation.cc:407] Connection header received
0806 14:59:05.340594 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:05.340595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:05.340606 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:05.340668 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:05.341007 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:05.341648 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:05.342535 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:05.342771 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:05.342861 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:05.345630 (+ 2769us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.65:57443: BlockingRecv error: recv got EOF from 127.11.8.65:57443 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":57,"threads_started":1}
Aug 06 14:59:05 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 14:59:05 dist-test-slave-68xk krb5kdc[22524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.11.8.66@KRBTEST.COM for oryx/127.11.8.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 14:59:05.457381 24118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:05.452457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:45081 (local address 127.11.8.66:60219)
0806 14:59:05.452647 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:05.452665 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:05.452760 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:05.452948 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:05.452949 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:05.453184 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:05.453248 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:05.453251 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:05.453923 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:05.453924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:05.454479 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:05.454481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:05.454839 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:05.454840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:05.454887 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:05.455450 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:59:05.455455 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:05.457319 (+ 1864us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.11.8.126:45081: Server oryx/127.11.8.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":103,"threads_started":1}
W20260806 14:59:05.457538 24117 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:59:05.452514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:60219 (local address 127.11.8.126:45081)
0806 14:59:05.452608 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:05.452611 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:05.452763 (+ 152us) server_negotiation.cc:407] Connection header received
0806 14:59:05.452834 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:05.452835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:05.452852 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:05.452937 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:05.453330 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:05.453832 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:05.454558 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:05.454755 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:05.454819 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:05.457445 (+ 2626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.11.8.66:60219: BlockingRecv error: recv got EOF from 127.11.8.66:60219 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 14:59:05.506701 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 23425
I20260806 14:59:05.517993 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 23650
I20260806 14:59:05.529040 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 23838
I20260806 14:59:05.539916 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 23310
2026-08-06T14:59:05Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17730 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T14:59:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T14:59:05Z Disabled control of system clock
I20260806 14:59:05.571624 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:38437
--webserver_interface=127.11.8.126
--webserver_port=0
--builtin_ntp_servers=127.11.8.84:42417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:38437
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 14:59:05.654119 24127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:59:05.654332 24127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:59:05.654352 24127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:59:05.655638 24127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 14:59:05.655680 24127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:59:05.655694 24127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:59:05.655743 24127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 14:59:05.655768 24127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 14:59:05.657116 24127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:42417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:38437
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:38437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:59:05.657382 24127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:59:05.657689 24127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:59:05.662959 24135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:59:05.662992 24127 server_base.cc:1034] running on GCE node
W20260806 14:59:05.662974 24139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:59:05.662968 24136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:59:05.663497 24127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:59:05.663811 24127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:59:05.664973 24127 hybrid_clock.cc:648] HybridClock initialized: now 1786028345664962 us; error 32 us; skew 500 ppm
I20260806 14:59:05.666266 24127 webserver.cc:460] Webserver started at http://127.11.8.126:46749/ using document root <none> and password file <none>
I20260806 14:59:05.666559 24127 fs_manager.cc:359] Metadata directory not provided
I20260806 14:59:05.666608 24127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:59:05.666702 24127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:59:05.667693 24127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "ed0cf3312227464bac805c8d87c580bd"
format_stamp: "Formatted at 2026-08-06 14:59:05 on dist-test-slave-68xk"
server_key: "d21cb09816ca39f609a74a8f4a9aa47b"
server_key_iv: "5ec0c9aca67982c80b4312be1f7a9ec6"
server_key_version: "encryptionkey@0"
I20260806 14:59:05.667986 24127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "ed0cf3312227464bac805c8d87c580bd"
format_stamp: "Formatted at 2026-08-06 14:59:05 on dist-test-slave-68xk"
server_key: "d21cb09816ca39f609a74a8f4a9aa47b"
server_key_iv: "5ec0c9aca67982c80b4312be1f7a9ec6"
server_key_version: "encryptionkey@0"
I20260806 14:59:05.669268 24127 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 14:59:05.670007 24146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:05.670188 24127 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 14:59:05.670281 24127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "ed0cf3312227464bac805c8d87c580bd"
format_stamp: "Formatted at 2026-08-06 14:59:05 on dist-test-slave-68xk"
server_key: "d21cb09816ca39f609a74a8f4a9aa47b"
server_key_iv: "5ec0c9aca67982c80b4312be1f7a9ec6"
server_key_version: "encryptionkey@0"
I20260806 14:59:05.670338 24127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:59:05.692526 24127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:59:05.692844 24127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:59:05.692958 24127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:59:05.697211 24127 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:38437
I20260806 14:59:05.697278 24221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:38437 every 8 connection(s)
I20260806 14:59:05.697613 24127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 14:59:05.698525 24222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:59:05.701141 24222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed0cf3312227464bac805c8d87c580bd: Bootstrap starting.
I20260806 14:59:05.701751 24222 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed0cf3312227464bac805c8d87c580bd: Neither blocks nor log segments found. Creating new log.
I20260806 14:59:05.702059 24222 log.cc:824] T 00000000000000000000000000000000 P ed0cf3312227464bac805c8d87c580bd: Log is configured to *not* fsync() on all Append() calls
I20260806 14:59:05.702811 24222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed0cf3312227464bac805c8d87c580bd: No bootstrap required, opened a new log
I20260806 14:59:05.704351 24222 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed0cf3312227464bac805c8d87c580bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed0cf3312227464bac805c8d87c580bd" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38437 } }
I20260806 14:59:05.704514 24222 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed0cf3312227464bac805c8d87c580bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:59:05.704577 24222 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed0cf3312227464bac805c8d87c580bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed0cf3312227464bac805c8d87c580bd, State: Initialized, Role: FOLLOWER
I20260806 14:59:05.704764 24222 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed0cf3312227464bac805c8d87c580bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed0cf3312227464bac805c8d87c580bd" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38437 } }
I20260806 14:59:05.704872 24222 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed0cf3312227464bac805c8d87c580bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 14:59:05.704916 24222 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed0cf3312227464bac805c8d87c580bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 14:59:05.704965 24222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed0cf3312227464bac805c8d87c580bd [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:59:05.705513 24222 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed0cf3312227464bac805c8d87c580bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed0cf3312227464bac805c8d87c580bd" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38437 } }
I20260806 14:59:05.705632 24222 leader_election.cc:302] T 00000000000000000000000000000000 P ed0cf3312227464bac805c8d87c580bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed0cf3312227464bac805c8d87c580bd; no voters:
I20260806 14:59:05.705827 24222 leader_election.cc:288] T 00000000000000000000000000000000 P ed0cf3312227464bac805c8d87c580bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 14:59:05.705881 24231 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed0cf3312227464bac805c8d87c580bd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:59:05.706229 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 24127
I20260806 14:59:05.706331 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 14:59:05.706084 24222 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed0cf3312227464bac805c8d87c580bd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 14:59:05.706599 24231 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed0cf3312227464bac805c8d87c580bd [term 1 LEADER]: Becoming Leader. State: Replica: ed0cf3312227464bac805c8d87c580bd, State: Running, Role: LEADER
I20260806 14:59:05.706650 11297 external_mini_cluster.cc:1428] Setting key f8369ab23ce013dc238d60a560b08e51
I20260806 14:59:05.706795 24231 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed0cf3312227464bac805c8d87c580bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed0cf3312227464bac805c8d87c580bd" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38437 } }
I20260806 14:59:05.707397 24234 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed0cf3312227464bac805c8d87c580bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed0cf3312227464bac805c8d87c580bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed0cf3312227464bac805c8d87c580bd" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38437 } } }
I20260806 14:59:05.707571 24234 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed0cf3312227464bac805c8d87c580bd [sys.catalog]: This master's current role is: LEADER
I20260806 14:59:05.707834 24235 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed0cf3312227464bac805c8d87c580bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed0cf3312227464bac805c8d87c580bd. Latest consensus state: current_term: 1 leader_uuid: "ed0cf3312227464bac805c8d87c580bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed0cf3312227464bac805c8d87c580bd" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38437 } } }
I20260806 14:59:05.707916 24235 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed0cf3312227464bac805c8d87c580bd [sys.catalog]: This master's current role is: LEADER
I20260806 14:59:05.707989 24247 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 14:59:05.708555 24247 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 14:59:05.710379 24247 catalog_manager.cc:1347] Generated new cluster ID: 1c69aefc73c74903bc2f11a6259cff2d
I20260806 14:59:05.710462 24247 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 14:59:05.713797 24246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:05.707356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50592 (local address 127.11.8.126:38437)
0806 14:59:05.708120 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:05.708129 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:05.708163 (+ 34us) server_negotiation.cc:407] Connection header received
0806 14:59:05.708291 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:05.708298 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:05.708368 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:05.708513 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:05.708815 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:05.710055 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:05.711195 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:05.711831 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:05.711960 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:05.712898 (+ 938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:59:05.712902 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:59:05.712902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 14:59:05.713113 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:59:05.713154 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 14:59:05.713172 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:59:05.713230 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:05.713493 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 14:59:05.713574 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":97,"threads_started":1}
I20260806 14:59:05.720184 24247 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 14:59:05.720634 24247 catalog_manager.cc:1504] Loading token signing keys...
I20260806 14:59:05.727605 24247 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed0cf3312227464bac805c8d87c580bd: Generated new TSK 0
I20260806 14:59:05.727780 24247 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 14:59:05.730932 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:0
--local_ip_for_outbound_sockets=127.11.8.65
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:38437
--builtin_ntp_servers=127.11.8.84:42417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 14:59:05.818917 24265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:59:05.819126 24265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:59:05.819147 24265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:59:05.820304 24265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:59:05.820348 24265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:59:05.820372 24265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 14:59:05.821717 24265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:42417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:38437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24265
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:59:05.821970 24265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:59:05.822294 24265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:59:05.828634 24272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:59:05.828667 24265 server_base.cc:1034] running on GCE node
W20260806 14:59:05.828648 24274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:59:05.828671 24276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:59:05.829097 24265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:59:05.829365 24265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:59:05.830514 24265 hybrid_clock.cc:648] HybridClock initialized: now 1786028345830498 us; error 32 us; skew 500 ppm
I20260806 14:59:05.832064 24265 webserver.cc:460] Webserver started at http://127.11.8.65:39311/ using document root <none> and password file <none>
I20260806 14:59:05.832329 24265 fs_manager.cc:359] Metadata directory not provided
I20260806 14:59:05.832399 24265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:59:05.832568 24265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:59:05.833655 24265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/data/instance:
uuid: "e06c263daf6a4cd4affef834d03a293b"
format_stamp: "Formatted at 2026-08-06 14:59:05 on dist-test-slave-68xk"
server_key: "5b9f799b011e66ebe757fc17c290dba9"
server_key_iv: "4c9b827939e2a4d1a32abe35699032ca"
server_key_version: "encryptionkey@0"
I20260806 14:59:05.834030 24265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance:
uuid: "e06c263daf6a4cd4affef834d03a293b"
format_stamp: "Formatted at 2026-08-06 14:59:05 on dist-test-slave-68xk"
server_key: "5b9f799b011e66ebe757fc17c290dba9"
server_key_iv: "4c9b827939e2a4d1a32abe35699032ca"
server_key_version: "encryptionkey@0"
I20260806 14:59:05.835513 24265 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 14:59:05.836515 24285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:05.836850 24265 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:59:05.836958 24265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "e06c263daf6a4cd4affef834d03a293b"
format_stamp: "Formatted at 2026-08-06 14:59:05 on dist-test-slave-68xk"
server_key: "5b9f799b011e66ebe757fc17c290dba9"
server_key_iv: "4c9b827939e2a4d1a32abe35699032ca"
server_key_version: "encryptionkey@0"
I20260806 14:59:05.837033 24265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:59:05.859827 24265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:59:05.860136 24265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:59:05.860244 24265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:59:05.860447 24265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:59:05.860819 24265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:59:05.860855 24265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:05.860878 24265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:59:05.860930 24265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:05.867005 24265 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:45415
I20260806 14:59:05.867139 24443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:45415 every 8 connection(s)
I20260806 14:59:05.867457 24265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
I20260806 14:59:05.872745 24246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:05.868230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:34291 (local address 127.11.8.126:38437)
0806 14:59:05.868297 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:05.868299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:05.868549 (+ 250us) server_negotiation.cc:407] Connection header received
0806 14:59:05.869113 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:05.869115 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:05.869132 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:05.869185 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:05.869887 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:05.870400 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:05.871419 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:05.871612 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:05.871670 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:05.872503 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:59:05.872505 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:59:05.872505 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 14:59:05.872511 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:59:05.872525 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 14:59:05.872530 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:59:05.872574 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:05.872657 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 14:59:05.872684 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 14:59:05.872995 24451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:05.868211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38437 (local address 127.11.8.65:34291)
0806 14:59:05.868428 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:05.868462 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:05.868993 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:05.869256 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:05.869260 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:05.869271 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:05.869787 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:05.869792 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:05.870522 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:05.870523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:05.871331 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:05.871333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:05.871691 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:05.871691 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:05.871801 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:05.872357 (+ 556us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 14:59:05.872357 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:05.872401 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 14:59:05.872425 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:59:05.872586 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:59:05.872591 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:59:05.872593 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 14:59:05.872646 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 14:59:05.872729 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":102,"threads_started":1}
I20260806 14:59:05.873515 24449 heartbeater.cc:347] Connected to a master server at 127.11.8.126:38437
I20260806 14:59:05.873596 24449 heartbeater.cc:464] Registering TS with master...
I20260806 14:59:05.873737 24449 heartbeater.cc:511] Master 127.11.8.126:38437 requested a full tablet report, sending...
I20260806 14:59:05.874266 24170 ts_manager.cc:194] Registered new tserver with Master: e06c263daf6a4cd4affef834d03a293b (127.11.8.65:45415)
I20260806 14:59:05.874919 24170 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.8.65:34291
I20260806 14:59:05.877424 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 24265
I20260806 14:59:05.877542 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance
I20260806 14:59:05.877737 11297 external_mini_cluster.cc:1428] Setting key 71b553b12b344cc1cd7dd63de8baf183
I20260806 14:59:05.878702 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:0
--local_ip_for_outbound_sockets=127.11.8.66
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:38437
--builtin_ntp_servers=127.11.8.84:42417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 14:59:05.961440 24452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:59:05.961660 24452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:59:05.961681 24452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:59:05.962886 24452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:59:05.962929 24452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:59:05.962953 24452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 14:59:05.964303 24452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:42417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:38437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:59:05.964502 24452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:59:05.964761 24452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:59:05.969906 24460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:59:05.969897 24459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:59:05.970122 24452 server_base.cc:1034] running on GCE node
W20260806 14:59:05.969966 24462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:59:05.970376 24452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:59:05.970650 24452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:59:05.971832 24452 hybrid_clock.cc:648] HybridClock initialized: now 1786028345971815 us; error 35 us; skew 500 ppm
I20260806 14:59:05.973145 24452 webserver.cc:460] Webserver started at http://127.11.8.66:34335/ using document root <none> and password file <none>
I20260806 14:59:05.973434 24452 fs_manager.cc:359] Metadata directory not provided
I20260806 14:59:05.973510 24452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:59:05.973671 24452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:59:05.974850 24452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/data/instance:
uuid: "0d4f6d07db9e4387891fe34131710adf"
format_stamp: "Formatted at 2026-08-06 14:59:05 on dist-test-slave-68xk"
server_key: "e3520684b594797ab45d111d137e68d6"
server_key_iv: "fda18d39ce0ac463c9a81c30f1455059"
server_key_version: "encryptionkey@0"
I20260806 14:59:05.975251 24452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance:
uuid: "0d4f6d07db9e4387891fe34131710adf"
format_stamp: "Formatted at 2026-08-06 14:59:05 on dist-test-slave-68xk"
server_key: "e3520684b594797ab45d111d137e68d6"
server_key_iv: "fda18d39ce0ac463c9a81c30f1455059"
server_key_version: "encryptionkey@0"
I20260806 14:59:05.976569 24452 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 14:59:05.977290 24469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:05.977464 24452 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:05.977571 24452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "0d4f6d07db9e4387891fe34131710adf"
format_stamp: "Formatted at 2026-08-06 14:59:05 on dist-test-slave-68xk"
server_key: "e3520684b594797ab45d111d137e68d6"
server_key_iv: "fda18d39ce0ac463c9a81c30f1455059"
server_key_version: "encryptionkey@0"
I20260806 14:59:05.977648 24452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:59:05.991688 24452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:59:05.992002 24452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:59:05.992161 24452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:59:05.992394 24452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:59:05.992734 24452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:59:05.992791 24452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:05.992832 24452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:59:05.992861 24452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:05.998730 24452 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:36785
I20260806 14:59:05.998775 24640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:36785 every 8 connection(s)
I20260806 14:59:05.999099 24452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
I20260806 14:59:06.004261 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 24452
I20260806 14:59:06.004354 24246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:05.999784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:51599 (local address 127.11.8.126:38437)
0806 14:59:05.999857 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:05.999859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:06.000173 (+ 314us) server_negotiation.cc:407] Connection header received
0806 14:59:06.000670 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:06.000671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:06.000689 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:06.000743 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:06.001418 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:06.001908 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:06.002957 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:06.003138 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:06.003194 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:06.004047 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:59:06.004048 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:59:06.004049 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 14:59:06.004054 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:59:06.004070 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 14:59:06.004076 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:59:06.004134 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:06.004227 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 14:59:06.004257 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 14:59:06.004371 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance
I20260806 14:59:06.004545 24643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:05.999795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38437 (local address 127.11.8.66:51599)
0806 14:59:06.000024 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:06.000069 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:06.000579 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:06.000829 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:06.000833 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:06.000842 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:06.001341 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:06.001346 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:06.001984 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:06.001985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:06.002880 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:06.002883 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:06.003220 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:06.003221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:06.003326 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:06.003892 (+ 566us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 14:59:06.003893 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:06.003921 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 14:59:06.003943 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:59:06.004142 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:59:06.004146 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:59:06.004149 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 14:59:06.004207 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 14:59:06.004286 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
I20260806 14:59:06.004637 11297 external_mini_cluster.cc:1428] Setting key c9782cae9fbe53509e773b37395442fc
I20260806 14:59:06.005235 24641 heartbeater.cc:347] Connected to a master server at 127.11.8.126:38437
I20260806 14:59:06.005342 24641 heartbeater.cc:464] Registering TS with master...
I20260806 14:59:06.005518 24641 heartbeater.cc:511] Master 127.11.8.126:38437 requested a full tablet report, sending...
I20260806 14:59:06.005892 24169 ts_manager.cc:194] Registered new tserver with Master: 0d4f6d07db9e4387891fe34131710adf (127.11.8.66:36785)
I20260806 14:59:06.005928 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:0
--local_ip_for_outbound_sockets=127.11.8.67
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:38437
--builtin_ntp_servers=127.11.8.84:42417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 14:59:06.006328 24169 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.8.66:51599
W20260806 14:59:06.088850 24644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:59:06.089049 24644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:59:06.089078 24644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:59:06.090139 24644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:59:06.090171 24644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:59:06.090195 24644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 14:59:06.091529 24644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:42417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:38437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:59:06.091714 24644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:59:06.091948 24644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 14:59:06.096997 24644 server_base.cc:1034] running on GCE node
W20260806 14:59:06.096987 24650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:59:06.096952 24649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:59:06.096979 24654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:59:06.097383 24644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:59:06.097653 24644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:59:06.098840 24644 hybrid_clock.cc:648] HybridClock initialized: now 1786028346098815 us; error 37 us; skew 500 ppm
I20260806 14:59:06.100203 24644 webserver.cc:460] Webserver started at http://127.11.8.67:38377/ using document root <none> and password file <none>
I20260806 14:59:06.100452 24644 fs_manager.cc:359] Metadata directory not provided
I20260806 14:59:06.100525 24644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:59:06.100687 24644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:59:06.101832 24644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/data/instance:
uuid: "1d9f0539bd8d498cb5da100c675bc1f2"
format_stamp: "Formatted at 2026-08-06 14:59:06 on dist-test-slave-68xk"
server_key: "938048e56d579e70d5956937d4017f3b"
server_key_iv: "fef0b45a8bffdceaa8415abf47af4945"
server_key_version: "encryptionkey@0"
I20260806 14:59:06.102221 24644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance:
uuid: "1d9f0539bd8d498cb5da100c675bc1f2"
format_stamp: "Formatted at 2026-08-06 14:59:06 on dist-test-slave-68xk"
server_key: "938048e56d579e70d5956937d4017f3b"
server_key_iv: "fef0b45a8bffdceaa8415abf47af4945"
server_key_version: "encryptionkey@0"
I20260806 14:59:06.103592 24644 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:59:06.104516 24668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:06.104717 24644 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:59:06.104825 24644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "1d9f0539bd8d498cb5da100c675bc1f2"
format_stamp: "Formatted at 2026-08-06 14:59:06 on dist-test-slave-68xk"
server_key: "938048e56d579e70d5956937d4017f3b"
server_key_iv: "fef0b45a8bffdceaa8415abf47af4945"
server_key_version: "encryptionkey@0"
I20260806 14:59:06.104897 24644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:59:06.128582 24644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:59:06.128921 24644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:59:06.129081 24644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:59:06.129329 24644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:59:06.129732 24644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:59:06.129793 24644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:06.129837 24644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:59:06.129866 24644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:06.135739 24644 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:33785
I20260806 14:59:06.135784 24829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:33785 every 8 connection(s)
I20260806 14:59:06.136112 24644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
I20260806 14:59:06.141639 24246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:06.136859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:49671 (local address 127.11.8.126:38437)
0806 14:59:06.136945 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:06.136947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:06.137162 (+ 215us) server_negotiation.cc:407] Connection header received
0806 14:59:06.137742 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:06.137743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:06.137756 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:06.137815 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:06.138582 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:06.139089 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:06.140130 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:06.140348 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:06.140428 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:06.141337 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:59:06.141338 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:59:06.141339 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 14:59:06.141344 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:59:06.141359 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 14:59:06.141364 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:59:06.141431 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:06.141525 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 14:59:06.141559 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 14:59:06.141875 24834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:06.136826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38437 (local address 127.11.8.67:49671)
0806 14:59:06.137041 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:06.137075 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:06.137627 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:06.137891 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:06.137895 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:06.137906 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:06.138469 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:06.138476 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:06.139213 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:06.139214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:06.140034 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:06.140036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:06.140438 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:06.140439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:06.140588 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:06.141169 (+ 581us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 14:59:06.141169 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:06.141204 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 14:59:06.141228 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:59:06.141446 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:59:06.141451 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:59:06.141455 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 14:59:06.141525 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 14:59:06.141628 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
I20260806 14:59:06.142071 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 24644
I20260806 14:59:06.142166 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance
I20260806 14:59:06.142371 24830 heartbeater.cc:347] Connected to a master server at 127.11.8.126:38437
I20260806 14:59:06.142472 11297 external_mini_cluster.cc:1428] Setting key b9aa62cf477db45affbf431dfe2b5511
I20260806 14:59:06.142496 24830 heartbeater.cc:464] Registering TS with master...
I20260806 14:59:06.142706 24830 heartbeater.cc:511] Master 127.11.8.126:38437 requested a full tablet report, sending...
I20260806 14:59:06.143143 24169 ts_manager.cc:194] Registered new tserver with Master: 1d9f0539bd8d498cb5da100c675bc1f2 (127.11.8.67:33785)
I20260806 14:59:06.143540 24169 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.8.67:49671
I20260806 14:59:06.154399 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 14:59:06.156497 24246 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:59:06.155873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50606 (local address 127.11.8.126:38437)
0806 14:59:06.155941 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:06.155943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:06.156063 (+ 120us) server_negotiation.cc:407] Connection header received
0806 14:59:06.156168 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:06.156169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:06.156182 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:06.156237 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:06.156412 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50606: BlockingRecv error: recv got EOF from 127.0.0.1:50606 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 14:59:06.156595 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 24265
I20260806 14:59:06.168776 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 24452
I20260806 14:59:06.176605 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 24644
I20260806 14:59:06.188874 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 24127
2026-08-06T14:59:06Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (648 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T14:59:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T14:59:06Z Disabled control of system clock
I20260806 14:59:06.220471 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:42687
--webserver_interface=127.11.8.126
--webserver_port=0
--builtin_ntp_servers=127.11.8.84:35159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:42687
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 14:59:06.302904 24850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:59:06.303117 24850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:59:06.303138 24850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:59:06.304322 24850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 14:59:06.304368 24850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:59:06.304381 24850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:59:06.304394 24850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 14:59:06.304452 24850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 14:59:06.305824 24850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:35159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:42687
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:42687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:59:06.306087 24850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:59:06.306388 24850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:59:06.311266 24860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:59:06.311283 24857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:59:06.311276 24856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:59:06.311563 24850 server_base.cc:1034] running on GCE node
I20260806 14:59:06.311846 24850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:59:06.312098 24850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:59:06.313303 24850 hybrid_clock.cc:648] HybridClock initialized: now 1786028346313291 us; error 53 us; skew 500 ppm
I20260806 14:59:06.314594 24850 webserver.cc:460] Webserver started at http://127.11.8.126:43651/ using document root <none> and password file <none>
I20260806 14:59:06.314859 24850 fs_manager.cc:359] Metadata directory not provided
I20260806 14:59:06.314929 24850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:59:06.315085 24850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:59:06.316162 24850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "c05c9da0f1ed4bb7aa596348dbb3443c"
format_stamp: "Formatted at 2026-08-06 14:59:06 on dist-test-slave-68xk"
server_key: "590127087ac9b899a0a875bbfd643d9c"
server_key_iv: "bdab63d1322339c503e9f6abe5f63a5c"
server_key_version: "encryptionkey@0"
I20260806 14:59:06.316543 24850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "c05c9da0f1ed4bb7aa596348dbb3443c"
format_stamp: "Formatted at 2026-08-06 14:59:06 on dist-test-slave-68xk"
server_key: "590127087ac9b899a0a875bbfd643d9c"
server_key_iv: "bdab63d1322339c503e9f6abe5f63a5c"
server_key_version: "encryptionkey@0"
I20260806 14:59:06.317945 24850 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 14:59:06.318750 24868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:06.318926 24850 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:06.319034 24850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "c05c9da0f1ed4bb7aa596348dbb3443c"
format_stamp: "Formatted at 2026-08-06 14:59:06 on dist-test-slave-68xk"
server_key: "590127087ac9b899a0a875bbfd643d9c"
server_key_iv: "bdab63d1322339c503e9f6abe5f63a5c"
server_key_version: "encryptionkey@0"
I20260806 14:59:06.319108 24850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:59:06.322458 24850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:59:06.322738 24850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:59:06.322882 24850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:59:06.326869 24850 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:42687
I20260806 14:59:06.326898 24945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:42687 every 8 connection(s)
I20260806 14:59:06.327239 24850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 14:59:06.327899 24946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:59:06.330876 24946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c05c9da0f1ed4bb7aa596348dbb3443c: Bootstrap starting.
I20260806 14:59:06.331472 24946 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c05c9da0f1ed4bb7aa596348dbb3443c: Neither blocks nor log segments found. Creating new log.
I20260806 14:59:06.331737 24946 log.cc:824] T 00000000000000000000000000000000 P c05c9da0f1ed4bb7aa596348dbb3443c: Log is configured to *not* fsync() on all Append() calls
I20260806 14:59:06.332509 24946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c05c9da0f1ed4bb7aa596348dbb3443c: No bootstrap required, opened a new log
I20260806 14:59:06.334095 24946 raft_consensus.cc:348] T 00000000000000000000000000000000 P c05c9da0f1ed4bb7aa596348dbb3443c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c05c9da0f1ed4bb7aa596348dbb3443c" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 42687 } }
I20260806 14:59:06.334237 24946 raft_consensus.cc:374] T 00000000000000000000000000000000 P c05c9da0f1ed4bb7aa596348dbb3443c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:59:06.334260 24946 raft_consensus.cc:729] T 00000000000000000000000000000000 P c05c9da0f1ed4bb7aa596348dbb3443c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c05c9da0f1ed4bb7aa596348dbb3443c, State: Initialized, Role: FOLLOWER
I20260806 14:59:06.334407 24946 consensus_queue.cc:260] T 00000000000000000000000000000000 P c05c9da0f1ed4bb7aa596348dbb3443c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c05c9da0f1ed4bb7aa596348dbb3443c" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 42687 } }
I20260806 14:59:06.334534 24946 raft_consensus.cc:388] T 00000000000000000000000000000000 P c05c9da0f1ed4bb7aa596348dbb3443c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 14:59:06.334559 24946 raft_consensus.cc:482] T 00000000000000000000000000000000 P c05c9da0f1ed4bb7aa596348dbb3443c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 14:59:06.334589 24946 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c05c9da0f1ed4bb7aa596348dbb3443c [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:59:06.335040 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 24850
I20260806 14:59:06.335091 24946 raft_consensus.cc:504] T 00000000000000000000000000000000 P c05c9da0f1ed4bb7aa596348dbb3443c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c05c9da0f1ed4bb7aa596348dbb3443c" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 42687 } }
I20260806 14:59:06.335153 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 14:59:06.335227 24946 leader_election.cc:302] T 00000000000000000000000000000000 P c05c9da0f1ed4bb7aa596348dbb3443c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c05c9da0f1ed4bb7aa596348dbb3443c; no voters:
I20260806 14:59:06.335420 11297 external_mini_cluster.cc:1428] Setting key 732b0d2250e392b38a825f91d74e17b6
I20260806 14:59:06.335429 24946 leader_election.cc:288] T 00000000000000000000000000000000 P c05c9da0f1ed4bb7aa596348dbb3443c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 14:59:06.335467 24951 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c05c9da0f1ed4bb7aa596348dbb3443c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:59:06.335638 24951 raft_consensus.cc:686] T 00000000000000000000000000000000 P c05c9da0f1ed4bb7aa596348dbb3443c [term 1 LEADER]: Becoming Leader. State: Replica: c05c9da0f1ed4bb7aa596348dbb3443c, State: Running, Role: LEADER
I20260806 14:59:06.335695 24946 sys_catalog.cc:564] T 00000000000000000000000000000000 P c05c9da0f1ed4bb7aa596348dbb3443c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 14:59:06.335727 24951 consensus_queue.cc:237] T 00000000000000000000000000000000 P c05c9da0f1ed4bb7aa596348dbb3443c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c05c9da0f1ed4bb7aa596348dbb3443c" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 42687 } }
I20260806 14:59:06.336084 24952 sys_catalog.cc:455] T 00000000000000000000000000000000 P c05c9da0f1ed4bb7aa596348dbb3443c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c05c9da0f1ed4bb7aa596348dbb3443c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c05c9da0f1ed4bb7aa596348dbb3443c" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 42687 } } }
I20260806 14:59:06.336163 24952 sys_catalog.cc:458] T 00000000000000000000000000000000 P c05c9da0f1ed4bb7aa596348dbb3443c [sys.catalog]: This master's current role is: LEADER
I20260806 14:59:06.336319 24953 sys_catalog.cc:455] T 00000000000000000000000000000000 P c05c9da0f1ed4bb7aa596348dbb3443c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c05c9da0f1ed4bb7aa596348dbb3443c. Latest consensus state: current_term: 1 leader_uuid: "c05c9da0f1ed4bb7aa596348dbb3443c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c05c9da0f1ed4bb7aa596348dbb3443c" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 42687 } } }
I20260806 14:59:06.336392 24958 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 14:59:06.336469 24953 sys_catalog.cc:458] T 00000000000000000000000000000000 P c05c9da0f1ed4bb7aa596348dbb3443c [sys.catalog]: This master's current role is: LEADER
I20260806 14:59:06.337049 24958 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 14:59:06.338672 24955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:06.335905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42268 (local address 127.11.8.126:42687)
0806 14:59:06.336871 (+ 966us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:06.336882 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:06.336920 (+ 38us) server_negotiation.cc:407] Connection header received
0806 14:59:06.337034 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:06.337041 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:06.337087 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:06.337543 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:06.338154 (+ 611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:59:06.338157 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:59:06.338157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 14:59:06.338162 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:59:06.338207 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 14:59:06.338223 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:59:06.338266 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:06.338405 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 14:59:06.338485 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":889,"thread_start_us":62,"threads_started":1}
I20260806 14:59:06.338784 24958 catalog_manager.cc:1347] Generated new cluster ID: e30d686a8b754c85823d062157eb8dbf
I20260806 14:59:06.338861 24958 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 14:59:06.358089 24958 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 14:59:06.358330 24958 catalog_manager.cc:1504] Loading token signing keys...
I20260806 14:59:06.364006 24958 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c05c9da0f1ed4bb7aa596348dbb3443c: Generated new TSK 0
I20260806 14:59:06.364207 24958 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 14:59:06.369230 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:0
--local_ip_for_outbound_sockets=127.11.8.65
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:42687
--builtin_ntp_servers=127.11.8.84:35159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 14:59:06.457058 24986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:59:06.457314 24986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:59:06.457353 24986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:59:06.458622 24986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:59:06.458681 24986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:59:06.458725 24986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 14:59:06.460090 24986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:35159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:42687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:59:06.460368 24986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:59:06.460668 24986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:59:06.465804 24995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:59:06.465927 24986 server_base.cc:1034] running on GCE node
W20260806 14:59:06.465806 24996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:59:06.465797 24999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:59:06.466295 24986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:59:06.466565 24986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:59:06.467762 24986 hybrid_clock.cc:648] HybridClock initialized: now 1786028346467755 us; error 37 us; skew 500 ppm
I20260806 14:59:06.469071 24986 webserver.cc:460] Webserver started at http://127.11.8.65:35443/ using document root <none> and password file <none>
I20260806 14:59:06.469316 24986 fs_manager.cc:359] Metadata directory not provided
I20260806 14:59:06.469390 24986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:59:06.469545 24986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:59:06.470707 24986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/data/instance:
uuid: "27afa7e37913414cb0ecbb68beda682e"
format_stamp: "Formatted at 2026-08-06 14:59:06 on dist-test-slave-68xk"
server_key: "c81a6fc45281e04814d1c8b0746e87e9"
server_key_iv: "74ed18331fcb8e4238a2f6b2311d4792"
server_key_version: "encryptionkey@0"
I20260806 14:59:06.471067 24986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance:
uuid: "27afa7e37913414cb0ecbb68beda682e"
format_stamp: "Formatted at 2026-08-06 14:59:06 on dist-test-slave-68xk"
server_key: "c81a6fc45281e04814d1c8b0746e87e9"
server_key_iv: "74ed18331fcb8e4238a2f6b2311d4792"
server_key_version: "encryptionkey@0"
I20260806 14:59:06.472429 24986 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 14:59:06.473167 25008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:06.473361 24986 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 14:59:06.473469 24986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "27afa7e37913414cb0ecbb68beda682e"
format_stamp: "Formatted at 2026-08-06 14:59:06 on dist-test-slave-68xk"
server_key: "c81a6fc45281e04814d1c8b0746e87e9"
server_key_iv: "74ed18331fcb8e4238a2f6b2311d4792"
server_key_version: "encryptionkey@0"
I20260806 14:59:06.473542 24986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:59:06.476961 24986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:59:06.477223 24986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:59:06.477365 24986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:59:06.477566 24986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:59:06.477886 24986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:59:06.477941 24986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:06.477981 24986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:59:06.478010 24986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:06.483850 24986 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:39527
I20260806 14:59:06.483896 25170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:39527 every 8 connection(s)
I20260806 14:59:06.484210 24986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
I20260806 14:59:06.485103 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 24986
I20260806 14:59:06.485205 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance
I20260806 14:59:06.485437 11297 external_mini_cluster.cc:1428] Setting key e23045ee78abca623efbe29a5e44adc3
I20260806 14:59:06.486589 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:0
--local_ip_for_outbound_sockets=127.11.8.66
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:42687
--builtin_ntp_servers=127.11.8.84:35159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 14:59:06.487722 24955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:06.484966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:50905 (local address 127.11.8.126:42687)
0806 14:59:06.485054 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:06.485056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:06.485343 (+ 287us) server_negotiation.cc:407] Connection header received
0806 14:59:06.485959 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:06.485960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:06.485966 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:06.486011 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:06.487474 (+ 1463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:59:06.487475 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:59:06.487476 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 14:59:06.487480 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:59:06.487494 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 14:59:06.487498 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:59:06.487549 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:06.487656 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 14:59:06.487670 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 14:59:06.487951 25174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:06.484934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:42687 (local address 127.11.8.65:50905)
0806 14:59:06.485197 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:06.485243 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:06.485850 (+ 607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:06.486095 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:06.486099 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:06.486111 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:06.487303 (+ 1192us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 14:59:06.487304 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:06.487343 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 14:59:06.487379 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:59:06.487569 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:59:06.487575 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:59:06.487578 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 14:59:06.487655 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 14:59:06.487718 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":121,"threads_started":1}
I20260806 14:59:06.488481 25171 heartbeater.cc:347] Connected to a master server at 127.11.8.126:42687
I20260806 14:59:06.488590 25171 heartbeater.cc:464] Registering TS with master...
I20260806 14:59:06.488745 25171 heartbeater.cc:511] Master 127.11.8.126:42687 requested a full tablet report, sending...
I20260806 14:59:06.489244 24895 ts_manager.cc:194] Registered new tserver with Master: 27afa7e37913414cb0ecbb68beda682e (127.11.8.65:39527)
W20260806 14:59:06.570998 25176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:59:06.571214 25176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:59:06.571238 25176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:59:06.572423 25176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:59:06.572463 25176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:59:06.572489 25176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 14:59:06.573767 25176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:35159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:42687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:59:06.573964 25176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:59:06.574211 25176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:59:06.579412 25182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:59:06.579455 25183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:59:06.579457 25185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:59:06.579540 25176 server_base.cc:1034] running on GCE node
I20260806 14:59:06.579913 25176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:59:06.580171 25176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:59:06.581364 25176 hybrid_clock.cc:648] HybridClock initialized: now 1786028346581355 us; error 43 us; skew 500 ppm
I20260806 14:59:06.582837 25176 webserver.cc:460] Webserver started at http://127.11.8.66:43895/ using document root <none> and password file <none>
I20260806 14:59:06.583079 25176 fs_manager.cc:359] Metadata directory not provided
I20260806 14:59:06.583154 25176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:59:06.583303 25176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:59:06.584452 25176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/data/instance:
uuid: "6b254dee18024286b3a13d519c76a105"
format_stamp: "Formatted at 2026-08-06 14:59:06 on dist-test-slave-68xk"
server_key: "d20068c804a160ef213e806709979c3f"
server_key_iv: "5668c0e235eb086b30b54b9b52c686fa"
server_key_version: "encryptionkey@0"
I20260806 14:59:06.584824 25176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance:
uuid: "6b254dee18024286b3a13d519c76a105"
format_stamp: "Formatted at 2026-08-06 14:59:06 on dist-test-slave-68xk"
server_key: "d20068c804a160ef213e806709979c3f"
server_key_iv: "5668c0e235eb086b30b54b9b52c686fa"
server_key_version: "encryptionkey@0"
I20260806 14:59:06.586269 25176 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 14:59:06.587157 25194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:06.587395 25176 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:59:06.587496 25176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "6b254dee18024286b3a13d519c76a105"
format_stamp: "Formatted at 2026-08-06 14:59:06 on dist-test-slave-68xk"
server_key: "d20068c804a160ef213e806709979c3f"
server_key_iv: "5668c0e235eb086b30b54b9b52c686fa"
server_key_version: "encryptionkey@0"
I20260806 14:59:06.587582 25176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:59:06.591038 25176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:59:06.591307 25176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:59:06.591446 25176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:59:06.591667 25176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:59:06.591976 25176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:59:06.592029 25176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:06.592069 25176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:59:06.592097 25176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:06.598084 25176 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:43497
I20260806 14:59:06.598129 25367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:43497 every 8 connection(s)
I20260806 14:59:06.598528 25176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
I20260806 14:59:06.601560 24955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:06.599222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:53645 (local address 127.11.8.126:42687)
0806 14:59:06.599274 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:06.599277 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:06.599482 (+ 205us) server_negotiation.cc:407] Connection header received
0806 14:59:06.600010 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:06.600011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:06.600018 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:06.600088 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:06.601310 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:59:06.601312 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:59:06.601312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 14:59:06.601317 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:59:06.601335 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 14:59:06.601339 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:59:06.601399 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:06.601475 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 14:59:06.601490 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 14:59:06.601763 25371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:06.599179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:42687 (local address 127.11.8.66:53645)
0806 14:59:06.599346 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:06.599387 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:06.599893 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:06.600189 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:06.600203 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:06.600211 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:06.601152 (+ 941us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 14:59:06.601152 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:06.601181 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 14:59:06.601209 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:59:06.601406 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:59:06.601412 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:59:06.601414 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 14:59:06.601487 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 14:59:06.601554 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":69,"threads_started":1}
I20260806 14:59:06.602262 25368 heartbeater.cc:347] Connected to a master server at 127.11.8.126:42687
I20260806 14:59:06.602341 25368 heartbeater.cc:464] Registering TS with master...
I20260806 14:59:06.602535 25368 heartbeater.cc:511] Master 127.11.8.126:42687 requested a full tablet report, sending...
I20260806 14:59:06.602771 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 25176
I20260806 14:59:06.602880 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance
I20260806 14:59:06.602964 24895 ts_manager.cc:194] Registered new tserver with Master: 6b254dee18024286b3a13d519c76a105 (127.11.8.66:43497)
I20260806 14:59:06.603104 11297 external_mini_cluster.cc:1428] Setting key f82a42e22e8b4ac50b14aa4d23bdb615
I20260806 14:59:06.604009 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:0
--local_ip_for_outbound_sockets=127.11.8.67
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:42687
--builtin_ntp_servers=127.11.8.84:35159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 14:59:06.686058 25372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:59:06.686275 25372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:59:06.686297 25372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:59:06.687541 25372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:59:06.687587 25372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:59:06.687613 25372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 14:59:06.688951 25372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:35159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:42687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:59:06.689219 25372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:59:06.689514 25372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:59:06.694775 25382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:59:06.694762 25379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:59:06.694762 25380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:59:06.695007 25372 server_base.cc:1034] running on GCE node
I20260806 14:59:06.695291 25372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:59:06.695523 25372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:59:06.696686 25372 hybrid_clock.cc:648] HybridClock initialized: now 1786028346696678 us; error 34 us; skew 500 ppm
I20260806 14:59:06.697932 25372 webserver.cc:460] Webserver started at http://127.11.8.67:40529/ using document root <none> and password file <none>
I20260806 14:59:06.698196 25372 fs_manager.cc:359] Metadata directory not provided
I20260806 14:59:06.698246 25372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:59:06.698472 25372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:59:06.699576 25372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/data/instance:
uuid: "1cafbec587874777ba76d110488bf269"
format_stamp: "Formatted at 2026-08-06 14:59:06 on dist-test-slave-68xk"
server_key: "49d7f391d6aff09731c9a78e139907ca"
server_key_iv: "03e5de90fad93c040bb92ee08806c499"
server_key_version: "encryptionkey@0"
I20260806 14:59:06.699947 25372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance:
uuid: "1cafbec587874777ba76d110488bf269"
format_stamp: "Formatted at 2026-08-06 14:59:06 on dist-test-slave-68xk"
server_key: "49d7f391d6aff09731c9a78e139907ca"
server_key_iv: "03e5de90fad93c040bb92ee08806c499"
server_key_version: "encryptionkey@0"
I20260806 14:59:06.701279 25372 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 14:59:06.702018 25391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:06.702201 25372 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 14:59:06.702307 25372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "1cafbec587874777ba76d110488bf269"
format_stamp: "Formatted at 2026-08-06 14:59:06 on dist-test-slave-68xk"
server_key: "49d7f391d6aff09731c9a78e139907ca"
server_key_iv: "03e5de90fad93c040bb92ee08806c499"
server_key_version: "encryptionkey@0"
I20260806 14:59:06.702389 25372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:59:06.705693 25372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:59:06.705955 25372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:59:06.706102 25372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:59:06.706321 25372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:59:06.706656 25372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:59:06.706712 25372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:06.706750 25372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:59:06.706779 25372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:06.712815 25372 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:41821
I20260806 14:59:06.712860 25553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:41821 every 8 connection(s)
I20260806 14:59:06.713160 25372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
I20260806 14:59:06.716208 24955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:06.713848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:45933 (local address 127.11.8.126:42687)
0806 14:59:06.713895 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:06.713898 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:06.714117 (+ 219us) server_negotiation.cc:407] Connection header received
0806 14:59:06.714701 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:06.714702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:06.714710 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:06.714759 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:06.715938 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:59:06.715940 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:59:06.715941 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 14:59:06.715951 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:59:06.715964 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 14:59:06.715968 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:59:06.716032 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:06.716109 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 14:59:06.716126 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 14:59:06.716403 25557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:06.713800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:42687 (local address 127.11.8.67:45933)
0806 14:59:06.713986 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:06.714018 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:06.714581 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:06.714838 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:06.714842 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:06.714852 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:06.715777 (+ 925us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 14:59:06.715778 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:06.715805 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 14:59:06.715830 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:59:06.716029 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:59:06.716036 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:59:06.716040 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 14:59:06.716110 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 14:59:06.716192 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
I20260806 14:59:06.716920 25554 heartbeater.cc:347] Connected to a master server at 127.11.8.126:42687
I20260806 14:59:06.717013 25554 heartbeater.cc:464] Registering TS with master...
I20260806 14:59:06.717161 25554 heartbeater.cc:511] Master 127.11.8.126:42687 requested a full tablet report, sending...
I20260806 14:59:06.717559 24895 ts_manager.cc:194] Registered new tserver with Master: 1cafbec587874777ba76d110488bf269 (127.11.8.67:41821)
I20260806 14:59:06.719393 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 25372
I20260806 14:59:06.719472 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance
I20260806 14:59:06.719628 11297 external_mini_cluster.cc:1428] Setting key 63fdd9bbfc85dabd1be38da439b32de0
I20260806 14:59:06.720513 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 14:59:06.722491 24955 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 14:59:06.721905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42276 (local address 127.11.8.126:42687)
0806 14:59:06.721964 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:06.721967 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:06.722122 (+ 155us) server_negotiation.cc:407] Connection header received
0806 14:59:06.722146 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:06.722147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:06.722151 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:06.722209 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:06.722394 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42276: BlockingRecv error: recv got EOF from 127.0.0.1:42276 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 14:59:06.722510 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 24986
I20260806 14:59:06.739887 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 25176
I20260806 14:59:06.745148 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 25372
I20260806 14:59:06.759209 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 24850
2026-08-06T14:59:06Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (573 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 14:59:06 dist-test-slave-68xk krb5kdc[25568](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 14:59:06 dist-test-slave-68xk krb5kdc[25568](info): set up 2 sockets
Aug 06 14:59:06 dist-test-slave-68xk krb5kdc[25568](info): commencing operation
krb5kdc: starting...
W20260806 14:59:07.969472 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 14:59:08 dist-test-slave-68xk krb5kdc[25568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T14:59:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T14:59:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 14:59:08.093648 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:33925
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:34205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:33925
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:59:08.175426 25585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:59:08.175640 25585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:59:08.175666 25585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:59:08.176860 25585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 14:59:08.176906 25585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:59:08.176921 25585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:59:08.176934 25585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 14:59:08.176977 25585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 14:59:08.178570 25585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:34205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:33925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:33925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:59:08.178838 25585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:59:08.179070 25585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:59:08.184023 25598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:59:08.184072 25595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:59:08.184245 25585 server_base.cc:1034] running on GCE node
W20260806 14:59:08.184031 25594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:59:08.184556 25585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:59:08.184845 25585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:59:08.186017 25585 hybrid_clock.cc:648] HybridClock initialized: now 1786028348186009 us; error 40 us; skew 500 ppm
Aug 06 14:59:08 dist-test-slave-68xk krb5kdc[25568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028348, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:59:08.188665 25585 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 14:59:08.189148 25585 webserver.cc:460] Webserver started at http://127.11.8.126:36689/ using document root <none> and password file <none>
I20260806 14:59:08.189392 25585 fs_manager.cc:359] Metadata directory not provided
I20260806 14:59:08.189463 25585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:59:08.189615 25585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:59:08.190764 25585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "3855134ed91543699bf236025c5477d4"
format_stamp: "Formatted at 2026-08-06 14:59:08 on dist-test-slave-68xk"
server_key: "134ba084292cbc20223884ae55cca281"
server_key_iv: "1e734318d4241f4f829d6c905a21deb9"
server_key_version: "encryptionkey@0"
I20260806 14:59:08.191133 25585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "3855134ed91543699bf236025c5477d4"
format_stamp: "Formatted at 2026-08-06 14:59:08 on dist-test-slave-68xk"
server_key: "134ba084292cbc20223884ae55cca281"
server_key_iv: "1e734318d4241f4f829d6c905a21deb9"
server_key_version: "encryptionkey@0"
I20260806 14:59:08.192488 25585 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:59:08.193253 25609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:08.193428 25585 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 14:59:08.193537 25585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "3855134ed91543699bf236025c5477d4"
format_stamp: "Formatted at 2026-08-06 14:59:08 on dist-test-slave-68xk"
server_key: "134ba084292cbc20223884ae55cca281"
server_key_iv: "1e734318d4241f4f829d6c905a21deb9"
server_key_version: "encryptionkey@0"
I20260806 14:59:08.193612 25585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:59:08.205708 25585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:59:08.207821 25585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:59:08.207975 25585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:59:08.212486 25585 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:33925
I20260806 14:59:08.212522 25687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:33925 every 8 connection(s)
I20260806 14:59:08.212874 25585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 14:59:08.213920 25688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:59:08.216450 25688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3855134ed91543699bf236025c5477d4: Bootstrap starting.
I20260806 14:59:08.217037 25688 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3855134ed91543699bf236025c5477d4: Neither blocks nor log segments found. Creating new log.
I20260806 14:59:08.217315 25688 log.cc:824] T 00000000000000000000000000000000 P 3855134ed91543699bf236025c5477d4: Log is configured to *not* fsync() on all Append() calls
I20260806 14:59:08.218041 25688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3855134ed91543699bf236025c5477d4: No bootstrap required, opened a new log
I20260806 14:59:08.218508 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 25585
I20260806 14:59:08.218626 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 14:59:08.218880 11297 external_mini_cluster.cc:1428] Setting key 39618aae0306960a0812ae847fe688ab
I20260806 14:59:08.220472 25688 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3855134ed91543699bf236025c5477d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3855134ed91543699bf236025c5477d4" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 33925 } }
I20260806 14:59:08.220669 25688 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3855134ed91543699bf236025c5477d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:59:08.220717 25688 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3855134ed91543699bf236025c5477d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3855134ed91543699bf236025c5477d4, State: Initialized, Role: FOLLOWER
I20260806 14:59:08.220880 25688 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3855134ed91543699bf236025c5477d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3855134ed91543699bf236025c5477d4" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 33925 } }
I20260806 14:59:08.220981 25688 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3855134ed91543699bf236025c5477d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 14:59:08.221024 25688 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3855134ed91543699bf236025c5477d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 14:59:08.221077 25688 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3855134ed91543699bf236025c5477d4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:59:08.222177 25688 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3855134ed91543699bf236025c5477d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3855134ed91543699bf236025c5477d4" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 33925 } }
I20260806 14:59:08.222276 25688 leader_election.cc:302] T 00000000000000000000000000000000 P 3855134ed91543699bf236025c5477d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3855134ed91543699bf236025c5477d4; no voters:
I20260806 14:59:08.222532 25688 leader_election.cc:288] T 00000000000000000000000000000000 P 3855134ed91543699bf236025c5477d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 14:59:08.222553 25694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3855134ed91543699bf236025c5477d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:59:08.222685 25694 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3855134ed91543699bf236025c5477d4 [term 1 LEADER]: Becoming Leader. State: Replica: 3855134ed91543699bf236025c5477d4, State: Running, Role: LEADER
I20260806 14:59:08.222779 25694 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3855134ed91543699bf236025c5477d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3855134ed91543699bf236025c5477d4" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 33925 } }
I20260806 14:59:08.222836 25688 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3855134ed91543699bf236025c5477d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 14:59:08.223126 25694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3855134ed91543699bf236025c5477d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3855134ed91543699bf236025c5477d4. Latest consensus state: current_term: 1 leader_uuid: "3855134ed91543699bf236025c5477d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3855134ed91543699bf236025c5477d4" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 33925 } } }
I20260806 14:59:08.223209 25694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3855134ed91543699bf236025c5477d4 [sys.catalog]: This master's current role is: LEADER
I20260806 14:59:08.223361 25694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3855134ed91543699bf236025c5477d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3855134ed91543699bf236025c5477d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3855134ed91543699bf236025c5477d4" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 33925 } } }
I20260806 14:59:08.223518 25694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3855134ed91543699bf236025c5477d4 [sys.catalog]: This master's current role is: LEADER
I20260806 14:59:08.223912 25698 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 14:59:08.224565 25698 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 14:59:08.227232 25698 catalog_manager.cc:1347] Generated new cluster ID: dfca5b39469648288078eb96e32b9589
I20260806 14:59:08.227290 25698 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 14:59:08 dist-test-slave-68xk krb5kdc[25568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:59:08.232348 25693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:08.219483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38130 (local address 127.11.8.126:33925)
0806 14:59:08.219711 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:08.219720 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:08.219747 (+ 27us) server_negotiation.cc:407] Connection header received
0806 14:59:08.220448 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:08.220455 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:08.220504 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:08.220637 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:08.221539 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.222797 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.224083 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.224803 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.224921 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:08.228159 (+ 3238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:59:08.228164 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:59:08.228164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:59:08.228178 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:59:08.230495 (+ 2317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:59:08.230917 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:59:08.230921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:59:08.230922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:59:08.231105 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:59:08.231332 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:59:08.231333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:59:08.231334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:59:08.231677 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:59:08.231865 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:08.231945 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 14:59:08.232114 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":94,"threads_started":1}
I20260806 14:59:08.238999 25698 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 14:59:08.239570 25698 catalog_manager.cc:1504] Loading token signing keys...
I20260806 14:59:08.243494 25698 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3855134ed91543699bf236025c5477d4: Generated new TSK 0
I20260806 14:59:08.243680 25698 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
I20260806 14:59:08.273316 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:0
--local_ip_for_outbound_sockets=127.11.8.65
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:33925
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:34205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:59:08.355690 25720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:59:08.355881 25720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:59:08.355904 25720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:59:08.357038 25720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:59:08.357081 25720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:59:08.357107 25720 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 14:59:08.358583 25720 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:34205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:33925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:59:08.358839 25720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:59:08.359076 25720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:59:08.364324 25726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:59:08.364428 25730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:59:08.364341 25725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:59:08.364357 25720 server_base.cc:1034] running on GCE node
I20260806 14:59:08.364773 25720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:59:08.365016 25720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:59:08.366216 25720 hybrid_clock.cc:648] HybridClock initialized: now 1786028348366223 us; error 52 us; skew 500 ppm
Aug 06 14:59:08 dist-test-slave-68xk krb5kdc[25568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028348, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:59:08.368953 25720 init.cc:377] Logged in from keytab as kudu/127.11.8.65@KRBTEST.COM (short username kudu)
I20260806 14:59:08.369436 25720 webserver.cc:460] Webserver started at http://127.11.8.65:44859/ using document root <none> and password file <none>
I20260806 14:59:08.369647 25720 fs_manager.cc:359] Metadata directory not provided
I20260806 14:59:08.369691 25720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:59:08.369782 25720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:59:08.370860 25720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/data/instance:
uuid: "5e617618d2c148168bd71907469ad62d"
format_stamp: "Formatted at 2026-08-06 14:59:08 on dist-test-slave-68xk"
server_key: "083ad929e880a76dbf793658545bb32c"
server_key_iv: "59481bdacaec36c5c5aeac1e45ae6d62"
server_key_version: "encryptionkey@0"
I20260806 14:59:08.371167 25720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance:
uuid: "5e617618d2c148168bd71907469ad62d"
format_stamp: "Formatted at 2026-08-06 14:59:08 on dist-test-slave-68xk"
server_key: "083ad929e880a76dbf793658545bb32c"
server_key_iv: "59481bdacaec36c5c5aeac1e45ae6d62"
server_key_version: "encryptionkey@0"
I20260806 14:59:08.372480 25720 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 14:59:08.373257 25740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:08.373466 25720 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:59:08.373567 25720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "5e617618d2c148168bd71907469ad62d"
format_stamp: "Formatted at 2026-08-06 14:59:08 on dist-test-slave-68xk"
server_key: "083ad929e880a76dbf793658545bb32c"
server_key_iv: "59481bdacaec36c5c5aeac1e45ae6d62"
server_key_version: "encryptionkey@0"
I20260806 14:59:08.373631 25720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:59:08.385524 25720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:59:08.387548 25720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:59:08.387701 25720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:59:08.387887 25720 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:59:08.388223 25720 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:59:08.388262 25720 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:08.388284 25720 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:59:08.388334 25720 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:08.394332 25720 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:37299
I20260806 14:59:08.394426 25904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:37299 every 8 connection(s)
I20260806 14:59:08.394762 25720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
I20260806 14:59:08.398849 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 25720
I20260806 14:59:08.398963 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance
I20260806 14:59:08.399183 11297 external_mini_cluster.cc:1428] Setting key 2210f303c2aa8d4795531c727e719906
Aug 06 14:59:08 dist-test-slave-68xk krb5kdc[25568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028348, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:59:08.404897 25693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:08.395507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:60865 (local address 127.11.8.126:33925)
0806 14:59:08.395567 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:08.395570 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:08.395805 (+ 235us) server_negotiation.cc:407] Connection header received
0806 14:59:08.396360 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:08.396362 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:08.396375 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:08.396431 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:08.397418 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.397928 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.399021 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.399284 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.399351 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:08.401798 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:59:08.401800 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:59:08.401800 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:59:08.401808 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:59:08.403400 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:59:08.403824 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:59:08.403826 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:59:08.403826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:59:08.403890 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:59:08.404166 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:59:08.404167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:59:08.404167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:59:08.404324 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:59:08.404393 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:08.404759 (+ 366us) server_negotiation.cc:299] Negotiation successful
0806 14:59:08.404832 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 14:59:08.405222 25908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:08.395501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:33925 (local address 127.11.8.65:60865)
0806 14:59:08.395686 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:08.395721 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:08.396204 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:08.396508 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:08.396511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:08.397118 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:08.397302 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:08.397307 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.398007 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.398008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:08.398910 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:08.398912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.399380 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.399382 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:08.399543 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:08.400087 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:59:08.400097 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:08.401698 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:59:08.403506 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:59:08.403514 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:59:08.403516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:59:08.403746 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:59:08.403967 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:59:08.403968 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:59:08.403968 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:59:08.404090 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:59:08.404419 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:59:08.404427 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:59:08.404605 (+ 178us) client_negotiation.cc:769] Sending connection context
0806 14:59:08.404733 (+ 128us) client_negotiation.cc:240] Negotiation successful
0806 14:59:08.404907 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":100,"threads_started":1}
I20260806 14:59:08.405725 25905 heartbeater.cc:347] Connected to a master server at 127.11.8.126:33925
I20260806 14:59:08.405838 25905 heartbeater.cc:464] Registering TS with master...
I20260806 14:59:08.406009 25905 heartbeater.cc:511] Master 127.11.8.126:33925 requested a full tablet report, sending...
I20260806 14:59:08.406719 25632 ts_manager.cc:194] Registered new tserver with Master: 5e617618d2c148168bd71907469ad62d (127.11.8.65:37299)
I20260806 14:59:08.407344 25632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.65@KRBTEST.COM'} at 127.11.8.65:60865
WARNING: no policy specified for kudu/127.11.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
I20260806 14:59:08.425655 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:0
--local_ip_for_outbound_sockets=127.11.8.66
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:33925
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:34205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:59:08.509768 25913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:59:08.510007 25913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:59:08.510048 25913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:59:08.511293 25913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:59:08.511364 25913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:59:08.511409 25913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 14:59:08.512872 25913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:34205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:33925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:59:08.513151 25913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:59:08.513401 25913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:59:08.518932 25920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:59:08.518950 25919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:59:08.518913 25925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:59:08.519104 25913 server_base.cc:1034] running on GCE node
I20260806 14:59:08.519366 25913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:59:08.519636 25913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:59:08.520818 25913 hybrid_clock.cc:648] HybridClock initialized: now 1786028348520800 us; error 35 us; skew 500 ppm
Aug 06 14:59:08 dist-test-slave-68xk krb5kdc[25568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028348, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:59:08.523422 25913 init.cc:377] Logged in from keytab as kudu/127.11.8.66@KRBTEST.COM (short username kudu)
I20260806 14:59:08.523968 25913 webserver.cc:460] Webserver started at http://127.11.8.66:44101/ using document root <none> and password file <none>
I20260806 14:59:08.524281 25913 fs_manager.cc:359] Metadata directory not provided
I20260806 14:59:08.524328 25913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:59:08.524472 25913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:59:08.525558 25913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/data/instance:
uuid: "14dc391b9ad843609c38ed0b92fedfc0"
format_stamp: "Formatted at 2026-08-06 14:59:08 on dist-test-slave-68xk"
server_key: "776acb2da42a423a7b83851a06be08e0"
server_key_iv: "5a959324ac6cb6cf08c519999392d7f8"
server_key_version: "encryptionkey@0"
I20260806 14:59:08.525933 25913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance:
uuid: "14dc391b9ad843609c38ed0b92fedfc0"
format_stamp: "Formatted at 2026-08-06 14:59:08 on dist-test-slave-68xk"
server_key: "776acb2da42a423a7b83851a06be08e0"
server_key_iv: "5a959324ac6cb6cf08c519999392d7f8"
server_key_version: "encryptionkey@0"
I20260806 14:59:08.527415 25913 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 14:59:08.528290 25936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:08.528520 25913 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 14:59:08.528632 25913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "14dc391b9ad843609c38ed0b92fedfc0"
format_stamp: "Formatted at 2026-08-06 14:59:08 on dist-test-slave-68xk"
server_key: "776acb2da42a423a7b83851a06be08e0"
server_key_iv: "5a959324ac6cb6cf08c519999392d7f8"
server_key_version: "encryptionkey@0"
I20260806 14:59:08.528724 25913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:59:08.544490 25913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:59:08.546590 25913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:59:08.546761 25913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:59:08.547005 25913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:59:08.547326 25913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:59:08.547384 25913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:08.547430 25913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:59:08.547454 25913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:08.554175 25913 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:42459
I20260806 14:59:08.554232 26102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:42459 every 8 connection(s)
I20260806 14:59:08.554560 25913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
Aug 06 14:59:08 dist-test-slave-68xk krb5kdc[25568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028348, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:59:08.561374 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 25913
I20260806 14:59:08.561491 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance
I20260806 14:59:08.561764 11297 external_mini_cluster.cc:1428] Setting key 5d40e1078e00681051a9af302c9422ca
I20260806 14:59:08.564325 25693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:08.555289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:39089 (local address 127.11.8.126:33925)
0806 14:59:08.555370 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:08.555373 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:08.555531 (+ 158us) server_negotiation.cc:407] Connection header received
0806 14:59:08.556119 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:08.556120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:08.556131 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:08.556182 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:08.557092 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.557610 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.558718 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.558900 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.558953 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:08.561124 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:59:08.561126 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:59:08.561126 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:59:08.561132 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:59:08.562861 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:59:08.563272 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:59:08.563273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:59:08.563273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:59:08.563338 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:59:08.563599 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:59:08.563601 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:59:08.563601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:59:08.563753 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:59:08.563832 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:08.564119 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 14:59:08.564234 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 14:59:08.564476 26105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:08.555225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:33925 (local address 127.11.8.66:39089)
0806 14:59:08.555404 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:08.555441 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:08.556012 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:08.556250 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:08.556253 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:08.556817 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:08.557003 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:08.557007 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.557704 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.557705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:08.558641 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:08.558643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.558978 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.558979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:08.559083 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:08.559548 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:59:08.559557 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:08.561037 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:59:08.562948 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:59:08.562955 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:59:08.562958 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:59:08.563190 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:59:08.563416 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:59:08.563417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:59:08.563417 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:59:08.563529 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:59:08.563836 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:59:08.563844 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:59:08.563982 (+ 138us) client_negotiation.cc:769] Sending connection context
0806 14:59:08.564085 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 14:59:08.564244 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":98,"threads_started":1}
I20260806 14:59:08.565085 26103 heartbeater.cc:347] Connected to a master server at 127.11.8.126:33925
I20260806 14:59:08.565171 26103 heartbeater.cc:464] Registering TS with master...
I20260806 14:59:08.565423 26103 heartbeater.cc:511] Master 127.11.8.126:33925 requested a full tablet report, sending...
I20260806 14:59:08.565807 25632 ts_manager.cc:194] Registered new tserver with Master: 14dc391b9ad843609c38ed0b92fedfc0 (127.11.8.66:42459)
I20260806 14:59:08.566185 25632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.66@KRBTEST.COM'} at 127.11.8.66:39089
WARNING: no policy specified for kudu/127.11.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
I20260806 14:59:08.590190 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:0
--local_ip_for_outbound_sockets=127.11.8.67
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:33925
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:34205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:59:08.672255 26108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:59:08.672500 26108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:59:08.672554 26108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:59:08.673736 26108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:59:08.673806 26108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:59:08.673851 26108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 14:59:08.675436 26108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:34205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:33925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:59:08.675710 26108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:59:08.675943 26108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:59:08.681052 26118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:59:08.681120 26120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:59:08.681130 26117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:59:08.681507 26108 server_base.cc:1034] running on GCE node
I20260806 14:59:08.681748 26108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:59:08.681952 26108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:59:08.683077 26108 hybrid_clock.cc:648] HybridClock initialized: now 1786028348683055 us; error 38 us; skew 500 ppm
Aug 06 14:59:08 dist-test-slave-68xk krb5kdc[25568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028348, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:59:08.685621 26108 init.cc:377] Logged in from keytab as kudu/127.11.8.67@KRBTEST.COM (short username kudu)
I20260806 14:59:08.686003 26108 webserver.cc:460] Webserver started at http://127.11.8.67:37831/ using document root <none> and password file <none>
I20260806 14:59:08.686188 26108 fs_manager.cc:359] Metadata directory not provided
I20260806 14:59:08.686239 26108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:59:08.686329 26108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 14:59:08.687386 26108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/data/instance:
uuid: "8db7c0578bec4c3a9310b9efebf77a69"
format_stamp: "Formatted at 2026-08-06 14:59:08 on dist-test-slave-68xk"
server_key: "b0ce284f63b8b52c60d58e7aa7822237"
server_key_iv: "0a591ad8862d56eda1de870768a44c14"
server_key_version: "encryptionkey@0"
I20260806 14:59:08.687688 26108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance:
uuid: "8db7c0578bec4c3a9310b9efebf77a69"
format_stamp: "Formatted at 2026-08-06 14:59:08 on dist-test-slave-68xk"
server_key: "b0ce284f63b8b52c60d58e7aa7822237"
server_key_iv: "0a591ad8862d56eda1de870768a44c14"
server_key_version: "encryptionkey@0"
I20260806 14:59:08.688937 26108 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 14:59:08.689688 26130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:08.690009 26108 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 14:59:08.690119 26108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "8db7c0578bec4c3a9310b9efebf77a69"
format_stamp: "Formatted at 2026-08-06 14:59:08 on dist-test-slave-68xk"
server_key: "b0ce284f63b8b52c60d58e7aa7822237"
server_key_iv: "0a591ad8862d56eda1de870768a44c14"
server_key_version: "encryptionkey@0"
I20260806 14:59:08.690228 26108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:59:08.704239 26108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:59:08.706568 26108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:59:08.706746 26108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:59:08.707008 26108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:59:08.707374 26108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:59:08.707446 26108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:08.707497 26108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:59:08.707525 26108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:08.714002 26108 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:33257
I20260806 14:59:08.714039 26307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:33257 every 8 connection(s)
I20260806 14:59:08.714397 26108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
I20260806 14:59:08.715430 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 26108
I20260806 14:59:08.715564 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance
I20260806 14:59:08.715772 11297 external_mini_cluster.cc:1428] Setting key 9ae4026549929f064affa4508da8081d
Aug 06 14:59:08 dist-test-slave-68xk krb5kdc[25568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028348, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:59:08.724067 25693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:08.715205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:57321 (local address 127.11.8.126:33925)
0806 14:59:08.715273 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:08.715275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:08.715528 (+ 253us) server_negotiation.cc:407] Connection header received
0806 14:59:08.716186 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:08.716188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:08.716202 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:08.716281 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:08.717235 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.717732 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.718809 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.719015 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.719102 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:08.721237 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:59:08.721239 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:59:08.721240 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:59:08.721247 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:59:08.722665 (+ 1418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:59:08.723057 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:59:08.723058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:59:08.723059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:59:08.723116 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:59:08.723382 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:59:08.723383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:59:08.723384 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:59:08.723501 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:59:08.723575 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:08.723899 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 14:59:08.723978 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 14:59:08.724293 26313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:08.715209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:33925 (local address 127.11.8.67:57321)
0806 14:59:08.715374 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:08.715415 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:08.716064 (+ 649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:08.716353 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:08.716357 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:08.716946 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:08.717127 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:08.717131 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.717840 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.717841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:08.718715 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:08.718717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.719112 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.719113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:08.719243 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:08.719717 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:59:08.719725 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:08.721133 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:59:08.722753 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:59:08.722760 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:59:08.722762 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:59:08.722981 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:59:08.723207 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:59:08.723208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:59:08.723209 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:59:08.723313 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:59:08.723577 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:59:08.723584 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:59:08.723762 (+ 178us) client_negotiation.cc:769] Sending connection context
0806 14:59:08.723863 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 14:59:08.724017 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":90,"threads_started":1}
I20260806 14:59:08.724865 26309 heartbeater.cc:347] Connected to a master server at 127.11.8.126:33925
I20260806 14:59:08.724967 26309 heartbeater.cc:464] Registering TS with master...
I20260806 14:59:08.725152 26309 heartbeater.cc:511] Master 127.11.8.126:33925 requested a full tablet report, sending...
I20260806 14:59:08.725589 25632 ts_manager.cc:194] Registered new tserver with Master: 8db7c0578bec4c3a9310b9efebf77a69 (127.11.8.67:33257)
I20260806 14:59:08.725958 25632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.67@KRBTEST.COM'} at 127.11.8.67:57321
I20260806 14:59:08.727624 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 14:59:08.735056 25693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:08.728975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38146 (local address 127.11.8.126:33925)
0806 14:59:08.729040 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:08.729042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:08.729143 (+ 101us) server_negotiation.cc:407] Connection header received
0806 14:59:08.729179 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:08.729180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:08.729188 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:08.729259 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:08.729798 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.730260 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.731044 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.731239 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.731294 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:08.732579 (+ 1285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:59:08.732581 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:59:08.732581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:59:08.732595 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:59:08.733962 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:59:08.734346 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:59:08.734347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:59:08.734347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:59:08.734396 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:59:08.734651 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:59:08.734652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:59:08.734652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:59:08.734779 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:59:08.734855 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:08.734944 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 14:59:08.734998 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 14:59:08.736756 25632 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38146:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 14:59:08.738298 25632 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 14:59:08.744977 26326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:08.740570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:37299 (local address 127.0.0.1:48526)
0806 14:59:08.740900 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:08.740912 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:08.740978 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:08.741265 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:08.741268 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:08.741270 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:59:08.741337 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:08.741339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.742761 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.742762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:08.743785 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:08.743787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.744674 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.744675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:08.744741 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:08.744752 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 14:59:08.744843 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 14:59:08.744897 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":39,"thread_start_us":53,"threads_started":1}
I20260806 14:59:08.745002 26323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:08.740705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48526 (local address 127.11.8.65:37299)
0806 14:59:08.741100 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:08.741104 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:08.741112 (+ 8us) server_negotiation.cc:407] Connection header received
0806 14:59:08.741144 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:08.741145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:08.741157 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:08.741225 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:59:08.741574 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.742667 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.743872 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.744589 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.744670 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:08.744739 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:08.744873 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 14:59:08.744918 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":95,"threads_started":1}
I20260806 14:59:08.745285 26328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:08.740909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48264 (local address 127.11.8.66:42459)
0806 14:59:08.741101 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:08.741105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:08.741121 (+ 16us) server_negotiation.cc:407] Connection header received
0806 14:59:08.741162 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:08.741163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:08.741176 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:08.741278 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:59:08.741439 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.742606 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.744056 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.744740 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.744810 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:08.744884 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:08.745145 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 14:59:08.745202 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":85,"threads_started":1}
I20260806 14:59:08.745096 26322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:08.740570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:42459 (local address 127.0.0.1:48264)
0806 14:59:08.740728 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:08.740767 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:08.740828 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:08.741291 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:08.741292 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:08.741294 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:59:08.741344 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:08.741346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.742699 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.742700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:08.743975 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:08.743979 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.744820 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.744821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:08.744878 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:08.744882 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 14:59:08.744933 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 14:59:08.744969 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":84,"thread_start_us":63,"threads_started":1}
I20260806 14:59:08.746065 26321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:08.740765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:33257 (local address 127.0.0.1:34776)
0806 14:59:08.741382 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:08.741393 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:08.741448 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:08.741683 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:08.741683 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:08.741685 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:59:08.741721 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:08.741722 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.742642 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.742643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:08.743989 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:08.743991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.745827 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.745828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:08.745893 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:08.745898 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 14:59:08.745957 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 14:59:08.745998 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"mutex_wait_us":79,"thread_start_us":53,"threads_started":1}
I20260806 14:59:08.746096 26327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:08.740869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34776 (local address 127.11.8.67:33257)
0806 14:59:08.741031 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:08.741036 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:08.741445 (+ 409us) server_negotiation.cc:407] Connection header received
0806 14:59:08.741521 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:08.741523 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:08.741540 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:08.741608 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:59:08.741798 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.742537 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.744921 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.745665 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.745728 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:08.745789 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:08.745975 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 14:59:08.746020 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260806 14:59:08.746238 26005 tablet_service.cc:1467] Processing CreateTablet for tablet 07d649a8b04b4e10851009bd11c5e006 (DEFAULT_TABLE table=test-table [id=18b7761a105b4dd6b0e36b620142005d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:59:08.746658 26005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07d649a8b04b4e10851009bd11c5e006. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:59:08.747031 25809 tablet_service.cc:1467] Processing CreateTablet for tablet 07d649a8b04b4e10851009bd11c5e006 (DEFAULT_TABLE table=test-table [id=18b7761a105b4dd6b0e36b620142005d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:59:08.747116 26211 tablet_service.cc:1467] Processing CreateTablet for tablet 07d649a8b04b4e10851009bd11c5e006 (DEFAULT_TABLE table=test-table [id=18b7761a105b4dd6b0e36b620142005d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:59:08.747363 25809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07d649a8b04b4e10851009bd11c5e006. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:59:08.747435 26211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07d649a8b04b4e10851009bd11c5e006. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:59:08.750204 26329 tablet_bootstrap.cc:492] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0: Bootstrap starting.
I20260806 14:59:08.750568 26330 tablet_bootstrap.cc:492] T 07d649a8b04b4e10851009bd11c5e006 P 5e617618d2c148168bd71907469ad62d: Bootstrap starting.
I20260806 14:59:08.750734 26331 tablet_bootstrap.cc:492] T 07d649a8b04b4e10851009bd11c5e006 P 8db7c0578bec4c3a9310b9efebf77a69: Bootstrap starting.
I20260806 14:59:08.751210 26330 tablet_bootstrap.cc:654] T 07d649a8b04b4e10851009bd11c5e006 P 5e617618d2c148168bd71907469ad62d: Neither blocks nor log segments found. Creating new log.
I20260806 14:59:08.751221 26329 tablet_bootstrap.cc:654] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0: Neither blocks nor log segments found. Creating new log.
I20260806 14:59:08.751425 26331 tablet_bootstrap.cc:654] T 07d649a8b04b4e10851009bd11c5e006 P 8db7c0578bec4c3a9310b9efebf77a69: Neither blocks nor log segments found. Creating new log.
I20260806 14:59:08.751852 26330 log.cc:824] T 07d649a8b04b4e10851009bd11c5e006 P 5e617618d2c148168bd71907469ad62d: Log is configured to *not* fsync() on all Append() calls
I20260806 14:59:08.751852 26329 log.cc:824] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0: Log is configured to *not* fsync() on all Append() calls
I20260806 14:59:08.751890 26331 log.cc:824] T 07d649a8b04b4e10851009bd11c5e006 P 8db7c0578bec4c3a9310b9efebf77a69: Log is configured to *not* fsync() on all Append() calls
I20260806 14:59:08.752664 26331 tablet_bootstrap.cc:492] T 07d649a8b04b4e10851009bd11c5e006 P 8db7c0578bec4c3a9310b9efebf77a69: No bootstrap required, opened a new log
I20260806 14:59:08.752745 26331 ts_tablet_manager.cc:1397] T 07d649a8b04b4e10851009bd11c5e006 P 8db7c0578bec4c3a9310b9efebf77a69: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 14:59:08.752885 26330 tablet_bootstrap.cc:492] T 07d649a8b04b4e10851009bd11c5e006 P 5e617618d2c148168bd71907469ad62d: No bootstrap required, opened a new log
I20260806 14:59:08.752948 26330 ts_tablet_manager.cc:1397] T 07d649a8b04b4e10851009bd11c5e006 P 5e617618d2c148168bd71907469ad62d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 14:59:08.754793 26329 tablet_bootstrap.cc:492] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0: No bootstrap required, opened a new log
I20260806 14:59:08.754805 26330 raft_consensus.cc:348] T 07d649a8b04b4e10851009bd11c5e006 P 5e617618d2c148168bd71907469ad62d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e617618d2c148168bd71907469ad62d" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37299 } } peers { permanent_uuid: "14dc391b9ad843609c38ed0b92fedfc0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 42459 } } peers { permanent_uuid: "8db7c0578bec4c3a9310b9efebf77a69" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 33257 } }
I20260806 14:59:08.754778 26331 raft_consensus.cc:348] T 07d649a8b04b4e10851009bd11c5e006 P 8db7c0578bec4c3a9310b9efebf77a69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e617618d2c148168bd71907469ad62d" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37299 } } peers { permanent_uuid: "14dc391b9ad843609c38ed0b92fedfc0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 42459 } } peers { permanent_uuid: "8db7c0578bec4c3a9310b9efebf77a69" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 33257 } }
I20260806 14:59:08.754902 26329 ts_tablet_manager.cc:1397] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 14:59:08.754949 26330 raft_consensus.cc:374] T 07d649a8b04b4e10851009bd11c5e006 P 5e617618d2c148168bd71907469ad62d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:59:08.754949 26331 raft_consensus.cc:374] T 07d649a8b04b4e10851009bd11c5e006 P 8db7c0578bec4c3a9310b9efebf77a69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:59:08.754993 26331 raft_consensus.cc:729] T 07d649a8b04b4e10851009bd11c5e006 P 8db7c0578bec4c3a9310b9efebf77a69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8db7c0578bec4c3a9310b9efebf77a69, State: Initialized, Role: FOLLOWER
I20260806 14:59:08.754993 26330 raft_consensus.cc:729] T 07d649a8b04b4e10851009bd11c5e006 P 5e617618d2c148168bd71907469ad62d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e617618d2c148168bd71907469ad62d, State: Initialized, Role: FOLLOWER
I20260806 14:59:08.755105 26331 consensus_queue.cc:260] T 07d649a8b04b4e10851009bd11c5e006 P 8db7c0578bec4c3a9310b9efebf77a69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e617618d2c148168bd71907469ad62d" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37299 } } peers { permanent_uuid: "14dc391b9ad843609c38ed0b92fedfc0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 42459 } } peers { permanent_uuid: "8db7c0578bec4c3a9310b9efebf77a69" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 33257 } }
I20260806 14:59:08.755120 26330 consensus_queue.cc:260] T 07d649a8b04b4e10851009bd11c5e006 P 5e617618d2c148168bd71907469ad62d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e617618d2c148168bd71907469ad62d" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37299 } } peers { permanent_uuid: "14dc391b9ad843609c38ed0b92fedfc0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 42459 } } peers { permanent_uuid: "8db7c0578bec4c3a9310b9efebf77a69" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 33257 } }
I20260806 14:59:08.755388 26330 ts_tablet_manager.cc:1428] T 07d649a8b04b4e10851009bd11c5e006 P 5e617618d2c148168bd71907469ad62d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 14:59:08.755388 26331 ts_tablet_manager.cc:1428] T 07d649a8b04b4e10851009bd11c5e006 P 8db7c0578bec4c3a9310b9efebf77a69: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 14:59:08.755539 26309 heartbeater.cc:503] Master 127.11.8.126:33925 was elected leader, sending a full tablet report...
I20260806 14:59:08.755542 25905 heartbeater.cc:503] Master 127.11.8.126:33925 was elected leader, sending a full tablet report...
I20260806 14:59:08.757015 26329 raft_consensus.cc:348] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e617618d2c148168bd71907469ad62d" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37299 } } peers { permanent_uuid: "14dc391b9ad843609c38ed0b92fedfc0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 42459 } } peers { permanent_uuid: "8db7c0578bec4c3a9310b9efebf77a69" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 33257 } }
I20260806 14:59:08.757164 26329 raft_consensus.cc:374] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:59:08.757210 26329 raft_consensus.cc:729] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14dc391b9ad843609c38ed0b92fedfc0, State: Initialized, Role: FOLLOWER
I20260806 14:59:08.757319 26329 consensus_queue.cc:260] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e617618d2c148168bd71907469ad62d" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37299 } } peers { permanent_uuid: "14dc391b9ad843609c38ed0b92fedfc0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 42459 } } peers { permanent_uuid: "8db7c0578bec4c3a9310b9efebf77a69" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 33257 } }
I20260806 14:59:08.757602 26329 ts_tablet_manager.cc:1428] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 14:59:08.757670 26103 heartbeater.cc:503] Master 127.11.8.126:33925 was elected leader, sending a full tablet report...
W20260806 14:59:08.805198 26104 tablet.cc:2369] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 14:59:08.835973 26339 raft_consensus.cc:482] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 14:59:08.836136 26339 raft_consensus.cc:504] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e617618d2c148168bd71907469ad62d" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37299 } } peers { permanent_uuid: "14dc391b9ad843609c38ed0b92fedfc0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 42459 } } peers { permanent_uuid: "8db7c0578bec4c3a9310b9efebf77a69" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 33257 } }
I20260806 14:59:08.836534 26339 leader_election.cc:288] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e617618d2c148168bd71907469ad62d (127.11.8.65:37299), 8db7c0578bec4c3a9310b9efebf77a69 (127.11.8.67:33257)
I20260806 14:59:08.839745 26323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:08.836833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:37457 (local address 127.11.8.65:37299)
0806 14:59:08.836907 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:08.836909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:08.836918 (+ 9us) server_negotiation.cc:407] Connection header received
0806 14:59:08.836950 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:08.836951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:08.836966 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:08.837046 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:59:08.837371 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.837895 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.838986 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.839373 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.839464 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:08.839476 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:08.839629 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 14:59:08.839664 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 14:59:08.839745 26327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:08.836824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:45661 (local address 127.11.8.67:33257)
0806 14:59:08.836900 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:08.836903 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:08.837101 (+ 198us) server_negotiation.cc:407] Connection header received
0806 14:59:08.837175 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:08.837177 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:08.837192 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:08.837268 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:59:08.837483 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.837985 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.839009 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.839375 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.839464 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:08.839476 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:08.839632 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 14:59:08.839666 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 14:59:08.839781 26105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:08.836729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:37299 (local address 127.11.8.66:37457)
0806 14:59:08.836793 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:08.836806 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:08.836851 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:08.837171 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:08.837173 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:08.837176 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:59:08.837272 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:08.837274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.837981 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.837983 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:08.838884 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:08.838886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.839466 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.839467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:08.839549 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:08.839552 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 14:59:08.839621 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 14:59:08.839661 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":31}
I20260806 14:59:08.839802 26341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:08.836756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:33257 (local address 127.11.8.66:45661)
0806 14:59:08.836966 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:08.836977 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:08.837037 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:08.837289 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:08.837290 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:08.837292 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:59:08.837399 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:08.837401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.838076 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.838078 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:08.838897 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:08.838899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.839467 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.839468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:08.839548 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:08.839552 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 14:59:08.839623 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 14:59:08.839662 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":2,"thread_start_us":142,"threads_started":1}
I20260806 14:59:08.840170 25838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07d649a8b04b4e10851009bd11c5e006" candidate_uuid: "14dc391b9ad843609c38ed0b92fedfc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e617618d2c148168bd71907469ad62d" is_pre_election: true
I20260806 14:59:08.840199 26243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07d649a8b04b4e10851009bd11c5e006" candidate_uuid: "14dc391b9ad843609c38ed0b92fedfc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8db7c0578bec4c3a9310b9efebf77a69" is_pre_election: true
I20260806 14:59:08.840340 25838 raft_consensus.cc:2454] T 07d649a8b04b4e10851009bd11c5e006 P 5e617618d2c148168bd71907469ad62d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14dc391b9ad843609c38ed0b92fedfc0 in term 0.
I20260806 14:59:08.840372 26243 raft_consensus.cc:2454] T 07d649a8b04b4e10851009bd11c5e006 P 8db7c0578bec4c3a9310b9efebf77a69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14dc391b9ad843609c38ed0b92fedfc0 in term 0.
I20260806 14:59:08.840608 25940 leader_election.cc:302] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14dc391b9ad843609c38ed0b92fedfc0, 8db7c0578bec4c3a9310b9efebf77a69; no voters:
I20260806 14:59:08.840785 26339 raft_consensus.cc:2781] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 14:59:08.840876 26339 raft_consensus.cc:482] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 14:59:08.840898 26339 raft_consensus.cc:3037] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:59:08.841688 26339 raft_consensus.cc:504] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e617618d2c148168bd71907469ad62d" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37299 } } peers { permanent_uuid: "14dc391b9ad843609c38ed0b92fedfc0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 42459 } } peers { permanent_uuid: "8db7c0578bec4c3a9310b9efebf77a69" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 33257 } }
I20260806 14:59:08.841842 26339 leader_election.cc:288] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0 [CANDIDATE]: Term 1 election: Requested vote from peers 5e617618d2c148168bd71907469ad62d (127.11.8.65:37299), 8db7c0578bec4c3a9310b9efebf77a69 (127.11.8.67:33257)
I20260806 14:59:08.842012 26243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07d649a8b04b4e10851009bd11c5e006" candidate_uuid: "14dc391b9ad843609c38ed0b92fedfc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8db7c0578bec4c3a9310b9efebf77a69"
I20260806 14:59:08.842080 25838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07d649a8b04b4e10851009bd11c5e006" candidate_uuid: "14dc391b9ad843609c38ed0b92fedfc0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e617618d2c148168bd71907469ad62d"
I20260806 14:59:08.842121 26243 raft_consensus.cc:3037] T 07d649a8b04b4e10851009bd11c5e006 P 8db7c0578bec4c3a9310b9efebf77a69 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:59:08.842171 25838 raft_consensus.cc:3037] T 07d649a8b04b4e10851009bd11c5e006 P 5e617618d2c148168bd71907469ad62d [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:59:08.842960 26243 raft_consensus.cc:2454] T 07d649a8b04b4e10851009bd11c5e006 P 8db7c0578bec4c3a9310b9efebf77a69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14dc391b9ad843609c38ed0b92fedfc0 in term 1.
I20260806 14:59:08.843008 25838 raft_consensus.cc:2454] T 07d649a8b04b4e10851009bd11c5e006 P 5e617618d2c148168bd71907469ad62d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14dc391b9ad843609c38ed0b92fedfc0 in term 1.
I20260806 14:59:08.843191 25940 leader_election.cc:302] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14dc391b9ad843609c38ed0b92fedfc0, 8db7c0578bec4c3a9310b9efebf77a69; no voters:
I20260806 14:59:08.843310 26339 raft_consensus.cc:2781] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:59:08.843515 26339 raft_consensus.cc:686] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0 [term 1 LEADER]: Becoming Leader. State: Replica: 14dc391b9ad843609c38ed0b92fedfc0, State: Running, Role: LEADER
I20260806 14:59:08.843634 26339 consensus_queue.cc:237] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e617618d2c148168bd71907469ad62d" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37299 } } peers { permanent_uuid: "14dc391b9ad843609c38ed0b92fedfc0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 42459 } } peers { permanent_uuid: "8db7c0578bec4c3a9310b9efebf77a69" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 33257 } }
I20260806 14:59:08.844624 25631 catalog_manager.cc:5495] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 14dc391b9ad843609c38ed0b92fedfc0 (127.11.8.66). New cstate: current_term: 1 leader_uuid: "14dc391b9ad843609c38ed0b92fedfc0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e617618d2c148168bd71907469ad62d" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14dc391b9ad843609c38ed0b92fedfc0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 42459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8db7c0578bec4c3a9310b9efebf77a69" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 33257 } health_report { overall_health: UNKNOWN } } }
I20260806 14:59:08.863484 26328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:08.860999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48278 (local address 127.11.8.66:42459)
0806 14:59:08.861067 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:08.861069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:08.861097 (+ 28us) server_negotiation.cc:407] Connection header received
0806 14:59:08.861134 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:08.861135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:08.861156 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:08.861240 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 14:59:08.861352 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.861813 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.862684 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:08.862873 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:08.862928 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:08.863062 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 14:59:08.863165 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 14:59:08.863235 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:08.863326 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 14:59:08.863345 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 14:59:08.865607 25838 raft_consensus.cc:1261] T 07d649a8b04b4e10851009bd11c5e006 P 5e617618d2c148168bd71907469ad62d [term 1 FOLLOWER]: Refusing update from remote peer 14dc391b9ad843609c38ed0b92fedfc0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:59:08.865649 26243 raft_consensus.cc:1261] T 07d649a8b04b4e10851009bd11c5e006 P 8db7c0578bec4c3a9310b9efebf77a69 [term 1 FOLLOWER]: Refusing update from remote peer 14dc391b9ad843609c38ed0b92fedfc0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:59:08.866007 26339 consensus_queue.cc:1035] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e617618d2c148168bd71907469ad62d" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:59:08.866190 26342 consensus_queue.cc:1035] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8db7c0578bec4c3a9310b9efebf77a69" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 33257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:59:08.869444 26347 mvcc.cc:205] Tried to move back new op lower bound from 7315572116950331392 to 7315572116864135168. Current Snapshot: MvccSnapshot[applied={T|T < 7315572116950331392}]
I20260806 14:59:08.872174 25632 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:59:08.872298 25632 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:59:08.873090 25632 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3855134ed91543699bf236025c5477d4: Sending DeleteTablet for 3 replicas of tablet 07d649a8b04b4e10851009bd11c5e006
I20260806 14:59:08.873433 26005 tablet_service.cc:1514] Processing DeleteTablet for tablet 07d649a8b04b4e10851009bd11c5e006 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:59:08 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:48264
I20260806 14:59:08.873422 25809 tablet_service.cc:1514] Processing DeleteTablet for tablet 07d649a8b04b4e10851009bd11c5e006 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:59:08 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:48526
I20260806 14:59:08.873554 26211 tablet_service.cc:1514] Processing DeleteTablet for tablet 07d649a8b04b4e10851009bd11c5e006 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:59:08 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:34776
I20260806 14:59:08.873632 26359 tablet_replica.cc:330] T 07d649a8b04b4e10851009bd11c5e006 P 5e617618d2c148168bd71907469ad62d: stopping tablet replica
I20260806 14:59:08.873739 26359 raft_consensus.cc:2229] T 07d649a8b04b4e10851009bd11c5e006 P 5e617618d2c148168bd71907469ad62d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:59:08.873854 26359 raft_consensus.cc:2258] T 07d649a8b04b4e10851009bd11c5e006 P 5e617618d2c148168bd71907469ad62d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:59:08.874006 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 25720
I20260806 14:59:08.878549 26360 tablet_replica.cc:330] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0: stopping tablet replica
I20260806 14:59:08.878636 26360 raft_consensus.cc:2229] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0 [term 1 LEADER]: Raft consensus shutting down.
I20260806 14:59:08.878739 26360 raft_consensus.cc:2258] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:59:08.878898 26362 tablet_replica.cc:330] T 07d649a8b04b4e10851009bd11c5e006 P 8db7c0578bec4c3a9310b9efebf77a69: stopping tablet replica
I20260806 14:59:08.878989 26362 raft_consensus.cc:2229] T 07d649a8b04b4e10851009bd11c5e006 P 8db7c0578bec4c3a9310b9efebf77a69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:59:08.879076 26362 raft_consensus.cc:2258] T 07d649a8b04b4e10851009bd11c5e006 P 8db7c0578bec4c3a9310b9efebf77a69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:59:08.880437 26362 ts_tablet_manager.cc:1905] T 07d649a8b04b4e10851009bd11c5e006 P 8db7c0578bec4c3a9310b9efebf77a69: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:59:08.882694 26362 ts_tablet_manager.cc:1918] T 07d649a8b04b4e10851009bd11c5e006 P 8db7c0578bec4c3a9310b9efebf77a69: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 14:59:08.882777 26362 log.cc:1196] T 07d649a8b04b4e10851009bd11c5e006 P 8db7c0578bec4c3a9310b9efebf77a69: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-2/wal/wals/07d649a8b04b4e10851009bd11c5e006
I20260806 14:59:08.883077 26362 ts_tablet_manager.cc:1939] T 07d649a8b04b4e10851009bd11c5e006 P 8db7c0578bec4c3a9310b9efebf77a69: Deleting consensus metadata
I20260806 14:59:08.883354 26360 ts_tablet_manager.cc:1905] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:59:08.883961 25612 catalog_manager.cc:4841] TS 8db7c0578bec4c3a9310b9efebf77a69 (127.11.8.67:33257): tablet 07d649a8b04b4e10851009bd11c5e006 (table test-table [id=18b7761a105b4dd6b0e36b620142005d]) successfully deleted
I20260806 14:59:08.885529 26360 ts_tablet_manager.cc:1918] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 14:59:08.885617 26360 log.cc:1196] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786028287744255-11297-0/minicluster-data/ts-1/wal/wals/07d649a8b04b4e10851009bd11c5e006
I20260806 14:59:08.885910 26360 ts_tablet_manager.cc:1939] T 07d649a8b04b4e10851009bd11c5e006 P 14dc391b9ad843609c38ed0b92fedfc0: Deleting consensus metadata
I20260806 14:59:08.886507 25612 catalog_manager.cc:4841] TS 14dc391b9ad843609c38ed0b92fedfc0 (127.11.8.66:42459): tablet 07d649a8b04b4e10851009bd11c5e006 (table test-table [id=18b7761a105b4dd6b0e36b620142005d]) successfully deleted
W20260806 14:59:08.888415 25614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.65:37299 (error 108)
I20260806 14:59:08.888980 26321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:08.888827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:37299 (local address 127.0.0.1:48542)
0806 14:59:08.888897 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:08.888939 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.65:37299: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 14:59:08.889071 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 25913
W20260806 14:59:08.895035 25614 catalog_manager.cc:4584] TS 5e617618d2c148168bd71907469ad62d (127.11.8.65:37299): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 07d649a8b04b4e10851009bd11c5e006: Network error: Client connection negotiation failed: client connection to 127.11.8.65:37299: connect: Connection refused (error 111)
I20260806 14:59:08.900144 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 26108
I20260806 14:59:08.908890 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 25585
2026-08-06T14:59:08Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2148 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 14:59:08 dist-test-slave-68xk krb5kdc[26372](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 14:59:08 dist-test-slave-68xk krb5kdc[26372](info): set up 2 sockets
Aug 06 14:59:08 dist-test-slave-68xk krb5kdc[26372](info): commencing operation
krb5kdc: starting...
W20260806 14:59:10.140093 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 14:59:10 dist-test-slave-68xk krb5kdc[26372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T14:59:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T14:59:10Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.8.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.8.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.8.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/rangeradmin_127.11.8.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.8.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/rangeradmin_127.11.8.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.8.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.8.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.8.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/rangerlookup_127.11.8.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.8.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/rangerlookup_127.11.8.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.8.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.8.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.8.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/HTTP_127.11.8.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.8.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/HTTP_127.11.8.84@KRBTEST.COM.keytab.
I20260806 14:59:10.309371 11297 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskF1cjFt/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/postgres -l logfile start
2026-08-06 14:59:12.960 UTC [26415] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 14:59:12.960 UTC [26415] LOG: listening on IPv4 address "127.11.8.84", port 34889
2026-08-06 14:59:12.964 UTC [26415] LOG: listening on Unix socket "/tmp/.s.PGSQL.34889"
2026-08-06 14:59:12.987 UTC [26418] LOG: database system was shut down at 2026-08-06 14:59:11 UTC
2026-08-06 14:59:12.991 UTC [26415] LOG: database system is ready to accept connections
I20260806 14:59:14.143329 11297 mini_postgres.cc:96] Postgres bound to 34889
2026-08-06 14:59:14.151 UTC [26431] FATAL: database "slave" does not exist
127.11.8.84:34889 - accepting connections
I20260806 14:59:14.152218 11297 mini_ranger.cc:161] Starting Ranger...
I20260806 14:59:14.168475 11297 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 14:59:14.693712 11297 mini_ranger.cc:87] Created ranger Postgres database
I20260806 14:59:14.693792 11297 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ranger-admin
2026-08-06 14:59:15,034 [I] DB FLAVOR :POSTGRES
2026-08-06 14:59:15,035 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 14:59:15,035 [I] Checking connection..
2026-08-06 14:59:15,035 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:34889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 14:59:15,286 [I] Checking connection passed.
2026-08-06 14:59:15,286 [I] --------- Verifying version history table ---------
2026-08-06 14:59:15,286 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:34889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 14:59:15,560 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 14:59:15,560 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:34889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 14:59:15,808 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 14:59:15,808 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 14:59:15,808 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:34889/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 14:59:16.066 UTC [26528] WARNING: there is no transaction in progress
2026-08-06 14:59:16,072 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 14:59:16,073 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:34889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 14:59:16,337 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 14:59:16,337 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 14:59:16,337 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:34889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 14:59:16,596 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:34889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 14:59:16,829 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:34889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-68xk.c.gcp-upstream.internal','N') ;"
2026-08-06 14:59:17,102 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:34889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 14:59:17,383 [I] Table x_portal_user does not exist in database ranger
2026-08-06 14:59:17,384 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:34889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 14:59:17,633 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 14:59:17,633 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:34889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 14:59:17,873 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:34889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 14:59:18,116 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 14:59:18,117 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:34889/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 14:59:19.067 UTC [26679] WARNING: there is no transaction in progress
2026-08-06 14:59:19.084 UTC [26679] WARNING: there is no transaction in progress
2026-08-06 14:59:19.101 UTC [26679] WARNING: there is no transaction in progress
2026-08-06 14:59:19.117 UTC [26679] WARNING: there is no transaction in progress
2026-08-06 14:59:19.133 UTC [26679] WARNING: there is no transaction in progress
2026-08-06 14:59:19.151 UTC [26679] WARNING: there is no transaction in progress
2026-08-06 14:59:19.214 UTC [26679] WARNING: there is no transaction in progress
2026-08-06 14:59:19.222 UTC [26679] WARNING: there is no transaction in progress
2026-08-06 14:59:19.238 UTC [26679] WARNING: there is no transaction in progress
2026-08-06 14:59:19.249 UTC [26679] WARNING: there is no transaction in progress
2026-08-06 14:59:19.260 UTC [26679] WARNING: there is no transaction in progress
2026-08-06 14:59:19.275 UTC [26679] WARNING: there is no transaction in progress
2026-08-06 14:59:19.285 UTC [26679] WARNING: there is no transaction in progress
2026-08-06 14:59:19.292 UTC [26679] WARNING: there is no transaction in progress
2026-08-06 14:59:19.300 UTC [26679] WARNING: there is no transaction in progress
2026-08-06 14:59:19.307 UTC [26679] WARNING: there is no transaction in progress
2026-08-06 14:59:19,965 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 14:59:19,965 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:34889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 14:59:20,200 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 14:59:20,200 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:34889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 14:59:20,446 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 14:59:20,447 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:34889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 14:59:20,713 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 14:59:20,714 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:34889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 14:59:20,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:34889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 14:59:21,228 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:34889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-68xk.c.gcp-upstream.internal';"
2026-08-06 14:59:21,483 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 14:59:21,483 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:34889/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 14:59:21,726 [I] DB_PATCHES have already been applied
I20260806 14:59:21.731902 11297 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ranger-admin:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskF1cjFt/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 14:59:21.731971 11297 mini_ranger.cc:193] Using host: 127.11.8.84
I20260806 14:59:21.735762 11297 mini_ranger.cc:227] Ranger admin URL: http://127.11.8.84:40795
Aug 06, 2026 2:59:22 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 2:59:22 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 2:59:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 2:59:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 2:59:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 2:59:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 2:59:22 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 14:59:23.056849 11297 mini_ranger.cc:160] Time spent starting Ranger: real 8.905s user 0.002s sys 0.009s
Aug 06, 2026 2:59:43 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 2:59:43 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 2:59:43 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 2:59:43 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 2:59:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 2:59:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 2:59:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 2:59:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 2:59:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 2:59:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 2:59:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 2:59:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 2:59:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 2:59:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 2:59:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 2:59:43 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 2:59:43 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 2:59:44 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 14:59:45.412918 11297 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.8.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.8.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.8.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/rangerkms_127.11.8.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.8.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/rangerkms_127.11.8.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.8.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.8.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.8.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/HTTP_127.11.8.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.8.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/HTTP_127.11.8.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 14:59:45 dist-test-slave-68xk krb5kdc[26372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028385, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 14:59:45.500283 11297 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 14:59:45.511016 11297 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 14:59:46.426297 11297 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 14:59:46.426378 11297 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ranger-kms
I20260806 14:59:46.426393 11297 mini_ranger_kms.cc:227] Using postgres at 127.11.8.84:34889
2026-08-06 14:59:47,095 [I] DB FLAVOR :POSTGRES
2026-08-06 14:59:47,095 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 14:59:47,095 [I] Checking connection
2026-08-06 14:59:47,095 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:34889/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 14:59:47,336 [I] connection success
2026-08-06 14:59:47,337 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 14:59:47,337 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 14:59:47,337 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:34889/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 14:59:47,606 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 14:59:47,606 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 14:59:47,607 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 14:59:47,607 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:34889/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 14:59:47,866 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 14:59:48.352773 11297 mini_ranger_kms.cc:310] Created kms service
I20260806 14:59:48.559505 11297 mini_ranger_kms.cc:326] Created kudu user
I20260806 14:59:48.604673 11297 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 14:59:48 dist-test-slave-68xk krb5kdc[26372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028385, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.8.84@KRBTEST.COM
I20260806 14:59:49.054190 11297 mini_ranger_kms.cc:384] Added ranger policy
I20260806 14:59:49.054327 11297 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ranger-kms:/tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 14:59:49.054348 11297 mini_ranger_kms.cc:240] Using host: 127.11.8.84
I20260806 14:59:49.059093 11297 mini_ranger_kms.cc:276] Ranger KMS PID: 26938
I20260806 14:59:49.059222 11297 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.8.84:45467
Aug 06, 2026 2:59:50 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 2:59:50 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 2:59:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 2:59:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 2:59:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 2:59:50 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 14:59:51 dist-test-slave-68xk krb5kdc[26372](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786028391, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.8.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:59:51.753342 11297 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.253s user 0.016s sys 0.250s
I20260806 14:59:51.753461 11297 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 14:59:51.753548 11297 mini_ranger_kms.cc:401] 127.11.8.84:45467/kms/v1/keys
Aug 06 14:59:51 dist-test-slave-68xk krb5kdc[26372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028391, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.8.84@KRBTEST.COM for HTTP/127.11.8.84@KRBTEST.COM
Aug 06 14:59:52 dist-test-slave-68xk krb5kdc[26372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028391, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.8.84@KRBTEST.COM for HTTP/127.11.8.84@KRBTEST.COM
Aug 06 14:59:54 dist-test-slave-68xk krb5kdc[26372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 14:59:54.449589 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:36905
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:36513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:36905
--ranger_config_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.8.84:45467/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:59:54.592195 27006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:59:54.592492 27006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:59:54.592530 27006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:59:54.594195 27006 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 14:59:54.594260 27006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:59:54.594282 27006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:59:54.594301 27006 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 14:59:54.594317 27006 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 14:59:54.596943 27006 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:36513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.8.84:45467/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:36905
--ranger_config_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:36905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:59:54.597227 27006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:59:54.597483 27006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:59:54.603482 27017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:59:54.603648 27012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:59:54.603768 27011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:59:54.606825 27006 server_base.cc:1034] running on GCE node
I20260806 14:59:54.607091 27006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:59:54.607429 27006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:59:54.608613 27006 hybrid_clock.cc:648] HybridClock initialized: now 1786028394608601 us; error 32 us; skew 500 ppm
Aug 06 14:59:54 dist-test-slave-68xk krb5kdc[26372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028394, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:59:54.612293 27006 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 14:59:54.612793 27006 webserver.cc:460] Webserver started at http://127.11.8.126:41173/ using document root <none> and password file <none>
I20260806 14:59:54.613036 27006 fs_manager.cc:359] Metadata directory not provided
I20260806 14:59:54.613097 27006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:59:54.613262 27006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 14:59:54 dist-test-slave-68xk krb5kdc[26372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028394, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for HTTP/127.11.8.84@KRBTEST.COM
I20260806 14:59:54.675424 27006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "1ede0c91c35f4672bec06d01655036ad"
format_stamp: "Formatted at 2026-08-06 14:59:54 on dist-test-slave-68xk"
server_key: "c85fadf781846a854a9576348ad3949c"
server_key_iv: "97e2591f285c8bc78aebe365fb9d4ee0"
server_key_version: "kuduclusterkey@0"
I20260806 14:59:54.675832 27006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "1ede0c91c35f4672bec06d01655036ad"
format_stamp: "Formatted at 2026-08-06 14:59:54 on dist-test-slave-68xk"
server_key: "c85fadf781846a854a9576348ad3949c"
server_key_iv: "97e2591f285c8bc78aebe365fb9d4ee0"
server_key_version: "kuduclusterkey@0"
I20260806 14:59:54.677469 27006 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 14:59:54.692291 27025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:54.692581 27006 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 14:59:54.692698 27006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "1ede0c91c35f4672bec06d01655036ad"
format_stamp: "Formatted at 2026-08-06 14:59:54 on dist-test-slave-68xk"
server_key: "c85fadf781846a854a9576348ad3949c"
server_key_iv: "97e2591f285c8bc78aebe365fb9d4ee0"
server_key_version: "kuduclusterkey@0"
I20260806 14:59:54.692826 27006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:59:54.710546 27006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:59:54.712525 27006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:59:54.712720 27006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:59:54.725150 27006 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:36905
I20260806 14:59:54.725273 27100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:36905 every 8 connection(s)
I20260806 14:59:54.725785 27006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 14:59:54.726986 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 27006
I20260806 14:59:54.727092 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 14:59:54.727063 27101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:59:54.727286 11297 external_mini_cluster.cc:1428] Setting key e27587ddabae40af60bf5c1ea0f9beb6
I20260806 14:59:54.731956 27101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ede0c91c35f4672bec06d01655036ad: Bootstrap starting.
I20260806 14:59:54.733037 27101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ede0c91c35f4672bec06d01655036ad: Neither blocks nor log segments found. Creating new log.
I20260806 14:59:54.733944 27101 log.cc:824] T 00000000000000000000000000000000 P 1ede0c91c35f4672bec06d01655036ad: Log is configured to *not* fsync() on all Append() calls
I20260806 14:59:54.734952 27101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ede0c91c35f4672bec06d01655036ad: No bootstrap required, opened a new log
Aug 06 14:59:54 dist-test-slave-68xk krb5kdc[26372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:59:54.739179 27101 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ede0c91c35f4672bec06d01655036ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ede0c91c35f4672bec06d01655036ad" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 36905 } }
I20260806 14:59:54.739364 27101 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ede0c91c35f4672bec06d01655036ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:59:54.739428 27101 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ede0c91c35f4672bec06d01655036ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ede0c91c35f4672bec06d01655036ad, State: Initialized, Role: FOLLOWER
I20260806 14:59:54.739609 27101 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ede0c91c35f4672bec06d01655036ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ede0c91c35f4672bec06d01655036ad" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 36905 } }
I20260806 14:59:54.739710 27101 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ede0c91c35f4672bec06d01655036ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 14:59:54.739807 27101 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ede0c91c35f4672bec06d01655036ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 14:59:54.739868 27101 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ede0c91c35f4672bec06d01655036ad [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:59:54.740895 27101 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ede0c91c35f4672bec06d01655036ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ede0c91c35f4672bec06d01655036ad" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 36905 } }
I20260806 14:59:54.741008 27101 leader_election.cc:302] T 00000000000000000000000000000000 P 1ede0c91c35f4672bec06d01655036ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ede0c91c35f4672bec06d01655036ad; no voters:
I20260806 14:59:54.741307 27101 leader_election.cc:288] T 00000000000000000000000000000000 P 1ede0c91c35f4672bec06d01655036ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 14:59:54.741436 27109 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ede0c91c35f4672bec06d01655036ad [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:59:54.741717 27101 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ede0c91c35f4672bec06d01655036ad [sys.catalog]: configured and running, proceeding with master startup.
I20260806 14:59:54.741822 27109 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ede0c91c35f4672bec06d01655036ad [term 1 LEADER]: Becoming Leader. State: Replica: 1ede0c91c35f4672bec06d01655036ad, State: Running, Role: LEADER
I20260806 14:59:54.742019 27109 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ede0c91c35f4672bec06d01655036ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ede0c91c35f4672bec06d01655036ad" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 36905 } }
I20260806 14:59:54.742666 27109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ede0c91c35f4672bec06d01655036ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ede0c91c35f4672bec06d01655036ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ede0c91c35f4672bec06d01655036ad" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 36905 } } }
I20260806 14:59:54.742846 27109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ede0c91c35f4672bec06d01655036ad [sys.catalog]: This master's current role is: LEADER
I20260806 14:59:54.744309 27106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:54.728244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44698 (local address 127.11.8.126:36905)
0806 14:59:54.728673 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:54.728684 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:54.728754 (+ 70us) server_negotiation.cc:407] Connection header received
0806 14:59:54.729618 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:54.729629 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:54.729696 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:54.730056 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:54.731014 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:54.732779 (+ 1765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:54.734174 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:54.735023 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:54.735171 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:54.738788 (+ 3617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:59:54.738793 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:59:54.738793 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:59:54.738807 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:59:54.741323 (+ 2516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:59:54.741772 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:59:54.741775 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:59:54.741775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:59:54.741874 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:59:54.742113 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:59:54.742113 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:59:54.742114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:59:54.742466 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:59:54.743145 (+ 679us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:54.743214 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 14:59:54.743334 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":175,"threads_started":1}
I20260806 14:59:54.745504 27121 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 14:59:54.743709 27109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ede0c91c35f4672bec06d01655036ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ede0c91c35f4672bec06d01655036ad. Latest consensus state: current_term: 1 leader_uuid: "1ede0c91c35f4672bec06d01655036ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ede0c91c35f4672bec06d01655036ad" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 36905 } } }
I20260806 14:59:54.746397 27109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ede0c91c35f4672bec06d01655036ad [sys.catalog]: This master's current role is: LEADER
I20260806 14:59:54.747455 27121 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 14:59:54.747923 27101 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 14:59:54.749534 27121 catalog_manager.cc:1347] Generated new cluster ID: 5c0a645f1126495a9f35479d70b2c2f8
I20260806 14:59:54.749598 27121 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 14:59:54.764158 27121 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 14:59:54.764992 27121 catalog_manager.cc:1504] Loading token signing keys...
I20260806 14:59:54.782516 27121 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ede0c91c35f4672bec06d01655036ad: Generated new TSK 0
I20260806 14:59:54.782859 27121 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
I20260806 14:59:55.686465 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.8.84:45467/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:0
--local_ip_for_outbound_sockets=127.11.8.65
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:36905
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:36513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:59:55.859688 27167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:59:55.859953 27167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:59:55.859998 27167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:59:55.861248 27167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:59:55.861320 27167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:59:55.861367 27167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 14:59:55.863155 27167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:36513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.8.84:45467/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:36905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:59:55.863423 27167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:59:55.863683 27167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:59:55.871018 27177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:59:55.870996 27174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:59:55.871119 27173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:59:55.871331 27167 server_base.cc:1034] running on GCE node
I20260806 14:59:55.871677 27167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:59:55.871951 27167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:59:55.873144 27167 hybrid_clock.cc:648] HybridClock initialized: now 1786028395873122 us; error 42 us; skew 500 ppm
Aug 06 14:59:55 dist-test-slave-68xk krb5kdc[26372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028395, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:59:55.876883 27167 init.cc:377] Logged in from keytab as kudu/127.11.8.65@KRBTEST.COM (short username kudu)
I20260806 14:59:55.877611 27167 webserver.cc:460] Webserver started at http://127.11.8.65:38079/ using document root <none> and password file <none>
I20260806 14:59:55.877916 27167 fs_manager.cc:359] Metadata directory not provided
I20260806 14:59:55.877990 27167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:59:55.878140 27167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 14:59:55 dist-test-slave-68xk krb5kdc[26372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028395, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for HTTP/127.11.8.84@KRBTEST.COM
I20260806 14:59:55.899730 27167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-0/data/instance:
uuid: "2e20df9f557043e7b2fd47a443fec14e"
format_stamp: "Formatted at 2026-08-06 14:59:55 on dist-test-slave-68xk"
server_key: "7f86ba3f61c60a8e2f386411b4ddb995"
server_key_iv: "ef906c528258595419991e31fc99d6cd"
server_key_version: "kuduclusterkey@0"
I20260806 14:59:55.900352 27167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance:
uuid: "2e20df9f557043e7b2fd47a443fec14e"
format_stamp: "Formatted at 2026-08-06 14:59:55 on dist-test-slave-68xk"
server_key: "7f86ba3f61c60a8e2f386411b4ddb995"
server_key_iv: "ef906c528258595419991e31fc99d6cd"
server_key_version: "kuduclusterkey@0"
I20260806 14:59:55.903044 27167 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260806 14:59:55.922566 27187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:55.922806 27167 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 14:59:55.922905 27167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "2e20df9f557043e7b2fd47a443fec14e"
format_stamp: "Formatted at 2026-08-06 14:59:55 on dist-test-slave-68xk"
server_key: "7f86ba3f61c60a8e2f386411b4ddb995"
server_key_iv: "ef906c528258595419991e31fc99d6cd"
server_key_version: "kuduclusterkey@0"
I20260806 14:59:55.923007 27167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:59:55.943854 27167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:59:55.947527 27167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:59:55.947782 27167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:59:55.948241 27167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:59:55.948726 27167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:59:55.948768 27167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:55.948793 27167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:59:55.948807 27167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:55.969317 27167 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:37649
I20260806 14:59:55.969537 27356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:37649 every 8 connection(s)
I20260806 14:59:55.969853 27167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
I20260806 14:59:55.976799 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 27167
I20260806 14:59:55.976950 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance
I20260806 14:59:55.977219 11297 external_mini_cluster.cc:1428] Setting key 55ac90154bec20a405124e3b9ef793bf
Aug 06 14:59:55 dist-test-slave-68xk krb5kdc[26372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028395, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:59:55.985649 27360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:55.971433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:47341 (local address 127.11.8.126:36905)
0806 14:59:55.971569 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:55.971572 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:55.972046 (+ 474us) server_negotiation.cc:407] Connection header received
0806 14:59:55.973117 (+ 1071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:55.973118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:55.973135 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:55.973412 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:55.974487 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:55.975501 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:55.977126 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:55.977423 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:55.977470 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:55.981291 (+ 3821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:59:55.981293 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:59:55.981293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:59:55.981312 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:59:55.983594 (+ 2282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:59:55.984266 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:59:55.984267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:59:55.984267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:59:55.984360 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:59:55.984734 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:59:55.984736 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:59:55.984736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:59:55.984892 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:59:55.984944 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:55.985437 (+ 493us) server_negotiation.cc:299] Negotiation successful
0806 14:59:55.985536 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":66,"threads_started":1}
I20260806 14:59:55.985909 27361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:55.971374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:36905 (local address 127.11.8.65:47341)
0806 14:59:55.971730 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:55.971805 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:55.972986 (+ 1181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:55.973594 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:55.973599 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:55.973962 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:55.974325 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:55.974332 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:55.975637 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:55.975639 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:55.976989 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:55.976993 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:55.977707 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:55.977709 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:55.977857 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:55.979062 (+ 1205us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:59:55.979076 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:55.981166 (+ 2090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:59:55.983695 (+ 2529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:59:55.983703 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:59:55.983706 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:59:55.984102 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:59:55.984451 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:59:55.984452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:59:55.984452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:59:55.984619 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:59:55.985001 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:59:55.985008 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:59:55.985229 (+ 221us) client_negotiation.cc:769] Sending connection context
0806 14:59:55.985422 (+ 193us) client_negotiation.cc:240] Negotiation successful
0806 14:59:55.985632 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":206,"threads_started":1}
I20260806 14:59:55.986770 27358 heartbeater.cc:347] Connected to a master server at 127.11.8.126:36905
I20260806 14:59:55.986876 27358 heartbeater.cc:464] Registering TS with master...
I20260806 14:59:55.987089 27358 heartbeater.cc:511] Master 127.11.8.126:36905 requested a full tablet report, sending...
I20260806 14:59:55.987761 27049 ts_manager.cc:194] Registered new tserver with Master: 2e20df9f557043e7b2fd47a443fec14e (127.11.8.65:37649)
I20260806 14:59:55.988708 27049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.65@KRBTEST.COM'} at 127.11.8.65:47341
WARNING: no policy specified for kudu/127.11.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
I20260806 14:59:56.020062 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.8.84:45467/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:0
--local_ip_for_outbound_sockets=127.11.8.66
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:36905
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:36513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 14:59:56 dist-test-slave-68xk krb5kdc[26372](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786028396, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 14:59:56.188434 27364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:59:56.188671 27364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:59:56.188722 27364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:59:56.189961 27364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:59:56.190028 27364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:59:56.190073 27364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 14:59:56.193307 27364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:36513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.8.84:45467/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:36905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:59:56.193588 27364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:59:56.193975 27364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:59:56.202383 27373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:59:56.202049 27378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:59:56.202402 27375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:59:56.203948 27364 server_base.cc:1034] running on GCE node
I20260806 14:59:56.204219 27364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:59:56.204497 27364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:59:56.205730 27364 hybrid_clock.cc:648] HybridClock initialized: now 1786028396205703 us; error 41 us; skew 500 ppm
Aug 06 14:59:56 dist-test-slave-68xk krb5kdc[26372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028396, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:59:56.209654 27364 init.cc:377] Logged in from keytab as kudu/127.11.8.66@KRBTEST.COM (short username kudu)
I20260806 14:59:56.210518 27364 webserver.cc:460] Webserver started at http://127.11.8.66:37195/ using document root <none> and password file <none>
I20260806 14:59:56.210840 27364 fs_manager.cc:359] Metadata directory not provided
I20260806 14:59:56.210973 27364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:59:56.211164 27364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 14:59:56 dist-test-slave-68xk krb5kdc[26372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028396, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for HTTP/127.11.8.84@KRBTEST.COM
I20260806 14:59:56.234282 27364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-1/data/instance:
uuid: "775c6bb658f34326ad6bb2f24ee82b27"
format_stamp: "Formatted at 2026-08-06 14:59:56 on dist-test-slave-68xk"
server_key: "496335c0eb743dca80c5eedd323ad3e5"
server_key_iv: "9ef5f2949a8a834ec57d1a5bc03e185e"
server_key_version: "kuduclusterkey@0"
I20260806 14:59:56.234931 27364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance:
uuid: "775c6bb658f34326ad6bb2f24ee82b27"
format_stamp: "Formatted at 2026-08-06 14:59:56 on dist-test-slave-68xk"
server_key: "496335c0eb743dca80c5eedd323ad3e5"
server_key_iv: "9ef5f2949a8a834ec57d1a5bc03e185e"
server_key_version: "kuduclusterkey@0"
I20260806 14:59:56.238979 27364 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.001s
I20260806 14:59:56.254292 27389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:56.254571 27364 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260806 14:59:56.254688 27364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "775c6bb658f34326ad6bb2f24ee82b27"
format_stamp: "Formatted at 2026-08-06 14:59:56 on dist-test-slave-68xk"
server_key: "496335c0eb743dca80c5eedd323ad3e5"
server_key_iv: "9ef5f2949a8a834ec57d1a5bc03e185e"
server_key_version: "kuduclusterkey@0"
I20260806 14:59:56.254818 27364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:59:56.266685 27364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:59:56.269708 27364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:59:56.269946 27364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:59:56.270577 27364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:59:56.271165 27364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:59:56.271277 27364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:56.271366 27364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:59:56.271436 27364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:56.298085 27364 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:43667
I20260806 14:59:56.298830 27364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
I20260806 14:59:56.300758 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 27364
I20260806 14:59:56.300875 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance
I20260806 14:59:56.301132 11297 external_mini_cluster.cc:1428] Setting key 63491feac15e17e0aaefc4f71810f9cf
I20260806 14:59:56.315448 27557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:43667 every 8 connection(s)
Aug 06 14:59:56 dist-test-slave-68xk krb5kdc[26372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028396, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.8.67@KRBTEST.COM; defaulting to no policy
I20260806 14:59:56.333729 27360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:56.311768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:44155 (local address 127.11.8.126:36905)
0806 14:59:56.311819 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:56.311821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:56.320096 (+ 8275us) server_negotiation.cc:407] Connection header received
0806 14:59:56.320918 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:56.320920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:56.320935 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:56.320970 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:56.321989 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.322936 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.324552 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.324878 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.324961 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:56.329912 (+ 4951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:59:56.329914 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:59:56.329914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:59:56.329923 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:59:56.331865 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:59:56.332364 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:59:56.332366 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:59:56.332366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:59:56.332434 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:59:56.332747 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:59:56.332748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:59:56.332748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:59:56.332897 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:59:56.332970 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:56.333490 (+ 520us) server_negotiation.cc:299] Negotiation successful
0806 14:59:56.333591 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.67@KRBTEST.COM" created.
I20260806 14:59:56.335042 27567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:56.311280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:36905 (local address 127.11.8.66:44155)
0806 14:59:56.319835 (+ 8555us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:56.319905 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:56.320754 (+ 849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:56.321184 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:56.321190 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:56.321498 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:56.321823 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:56.321831 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.323061 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.323062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:56.324437 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:56.324440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.325055 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.325056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:56.325228 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:56.326154 (+ 926us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:59:56.326166 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:56.329786 (+ 3620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:59:56.331968 (+ 2182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:59:56.331976 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:59:56.331979 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:59:56.332287 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:59:56.332518 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:59:56.332519 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:59:56.332520 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:59:56.332667 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:59:56.333002 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:59:56.333009 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:59:56.333246 (+ 237us) client_negotiation.cc:769] Sending connection context
0806 14:59:56.334599 (+ 1353us) client_negotiation.cc:240] Negotiation successful
0806 14:59:56.334798 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8483,"thread_start_us":242,"threads_started":1}
I20260806 14:59:56.335857 27559 heartbeater.cc:347] Connected to a master server at 127.11.8.126:36905
I20260806 14:59:56.335956 27559 heartbeater.cc:464] Registering TS with master...
I20260806 14:59:56.336150 27559 heartbeater.cc:511] Master 127.11.8.126:36905 requested a full tablet report, sending...
I20260806 14:59:56.336513 27049 ts_manager.cc:194] Registered new tserver with Master: 775c6bb658f34326ad6bb2f24ee82b27 (127.11.8.66:43667)
I20260806 14:59:56.337014 27049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.66@KRBTEST.COM'} at 127.11.8.66:44155
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
I20260806 14:59:56.355660 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.8.84:45467/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:0
--local_ip_for_outbound_sockets=127.11.8.67
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:36905
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:36513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 14:59:56.524619 27569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 14:59:56.524909 27569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 14:59:56.524943 27569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 14:59:56.526670 27569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 14:59:56.526726 27569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 14:59:56.526851 27569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 14:59:56.529371 27569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:36513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.8.84:45467/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:36905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 14:59:56.529621 27569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 14:59:56.529875 27569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 14:59:56.538123 27577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 14:59:56.538133 27579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:59:56.538920 27569 server_base.cc:1034] running on GCE node
W20260806 14:59:56.539676 27576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 14:59:56.539965 27569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 14:59:56.540254 27569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 14:59:56.541428 27569 hybrid_clock.cc:648] HybridClock initialized: now 1786028396541403 us; error 42 us; skew 500 ppm
Aug 06 14:59:56 dist-test-slave-68xk krb5kdc[26372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028396, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 14:59:56.546314 27569 init.cc:377] Logged in from keytab as kudu/127.11.8.67@KRBTEST.COM (short username kudu)
I20260806 14:59:56.547041 27569 webserver.cc:460] Webserver started at http://127.11.8.67:34539/ using document root <none> and password file <none>
I20260806 14:59:56.547444 27569 fs_manager.cc:359] Metadata directory not provided
I20260806 14:59:56.547629 27569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 14:59:56.547895 27569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 14:59:56 dist-test-slave-68xk krb5kdc[26372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028396, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for HTTP/127.11.8.84@KRBTEST.COM
I20260806 14:59:56.573434 27569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-2/data/instance:
uuid: "d72656028f1c4d1fa81de87b56d926f0"
format_stamp: "Formatted at 2026-08-06 14:59:56 on dist-test-slave-68xk"
server_key: "59a3cc1adc673448d4f42ddd68bd4da4"
server_key_iv: "f04512990f69c14f08547215d823f4f0"
server_key_version: "kuduclusterkey@0"
I20260806 14:59:56.574384 27569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance:
uuid: "d72656028f1c4d1fa81de87b56d926f0"
format_stamp: "Formatted at 2026-08-06 14:59:56 on dist-test-slave-68xk"
server_key: "59a3cc1adc673448d4f42ddd68bd4da4"
server_key_iv: "f04512990f69c14f08547215d823f4f0"
server_key_version: "kuduclusterkey@0"
I20260806 14:59:56.576614 27569 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 14:59:56.596069 27591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:56.596325 27569 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260806 14:59:56.596429 27569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "d72656028f1c4d1fa81de87b56d926f0"
format_stamp: "Formatted at 2026-08-06 14:59:56 on dist-test-slave-68xk"
server_key: "59a3cc1adc673448d4f42ddd68bd4da4"
server_key_iv: "f04512990f69c14f08547215d823f4f0"
server_key_version: "kuduclusterkey@0"
I20260806 14:59:56.596575 27569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 14:59:56.617493 27569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 14:59:56.620613 27569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 14:59:56.620808 27569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 14:59:56.621091 27569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 14:59:56.621549 27569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 14:59:56.621598 27569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:56.621628 27569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 14:59:56.621645 27569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 14:59:56.648365 27569 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:34207
I20260806 14:59:56.648929 27569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
I20260806 14:59:56.650161 27744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:34207 every 8 connection(s)
I20260806 14:59:56.650669 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 27569
I20260806 14:59:56.650780 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance
I20260806 14:59:56.651002 11297 external_mini_cluster.cc:1428] Setting key 7389e630f64d1e62fede07f74297678e
Aug 06 14:59:56 dist-test-slave-68xk krb5kdc[26372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028396, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 14:59:56.665784 27360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:56.652018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:54015 (local address 127.11.8.126:36905)
0806 14:59:56.652138 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:56.652149 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:56.653434 (+ 1285us) server_negotiation.cc:407] Connection header received
0806 14:59:56.654209 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:56.654210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:56.654223 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:56.654252 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:56.655261 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.655997 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.657606 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.657969 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.658121 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:56.661415 (+ 3294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:59:56.661417 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:59:56.661418 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:59:56.661428 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:59:56.663349 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:59:56.664029 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:59:56.664031 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:59:56.664032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:59:56.664128 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:59:56.664602 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:59:56.664604 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:59:56.664615 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:59:56.664812 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:59:56.664880 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:56.665503 (+ 623us) server_negotiation.cc:299] Negotiation successful
0806 14:59:56.665634 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 14:59:56.666616 27754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:56.651439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:36905 (local address 127.11.8.67:54015)
0806 14:59:56.653150 (+ 1711us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:56.653200 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:56.654088 (+ 888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:56.654463 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:56.654468 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:56.654767 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0806 14:59:56.655104 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:56.655111 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.656124 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.656125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:56.657494 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:56.657497 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.658131 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.658132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:56.658344 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:56.659160 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 14:59:56.659172 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 14:59:56.661294 (+ 2122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 14:59:56.663461 (+ 2167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:59:56.663469 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:59:56.663472 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:59:56.663888 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:59:56.664255 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 14:59:56.664256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 14:59:56.664257 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 14:59:56.664457 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 14:59:56.664943 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 14:59:56.664951 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 14:59:56.665301 (+ 350us) client_negotiation.cc:769] Sending connection context
0806 14:59:56.665995 (+ 694us) client_negotiation.cc:240] Negotiation successful
0806 14:59:56.666242 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1656,"thread_start_us":321,"threads_started":1}
I20260806 14:59:56.667624 27747 heartbeater.cc:347] Connected to a master server at 127.11.8.126:36905
I20260806 14:59:56.667752 27747 heartbeater.cc:464] Registering TS with master...
I20260806 14:59:56.668004 27747 heartbeater.cc:511] Master 127.11.8.126:36905 requested a full tablet report, sending...
I20260806 14:59:56.668437 27049 ts_manager.cc:194] Registered new tserver with Master: d72656028f1c4d1fa81de87b56d926f0 (127.11.8.67:34207)
I20260806 14:59:56.669026 27049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.67@KRBTEST.COM'} at 127.11.8.67:54015
I20260806 14:59:56.674453 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 14:59:56.688699 27360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:56.676674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44702 (local address 127.11.8.126:36905)
0806 14:59:56.676716 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:56.676719 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:56.678900 (+ 2181us) server_negotiation.cc:407] Connection header received
0806 14:59:56.679009 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:56.679010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:56.679021 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:56.679051 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 14:59:56.679888 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.680654 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.681746 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.682018 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.682076 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:56.683715 (+ 1639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 14:59:56.683717 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 14:59:56.683717 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 14:59:56.683725 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 14:59:56.685379 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:59:56.687874 (+ 2495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:59:56.687875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:59:56.687876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:59:56.687936 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 14:59:56.688144 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 14:59:56.688145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 14:59:56.688145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 14:59:56.688266 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 14:59:56.688348 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:56.688536 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 14:59:56.688612 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 14:59:56.691430 27049 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44702:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 14:59:56.693871 27049 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 14:59:56.711990 27772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:56.705155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:34207 (local address 127.0.0.1:58524)
0806 14:59:56.706536 (+ 1381us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:56.706582 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:56.706661 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:56.707113 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:56.707116 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:56.707118 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:59:56.707186 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:56.707187 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.708943 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.708944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:56.710557 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:56.710560 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.711651 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.711652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:56.711780 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:56.711794 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 14:59:56.711860 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 14:59:56.711909 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1329,"thread_start_us":1075,"threads_started":1}
I20260806 14:59:56.717449 27770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:56.705413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58524 (local address 127.11.8.67:34207)
0806 14:59:56.706381 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:56.706385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:56.706809 (+ 424us) server_negotiation.cc:407] Connection header received
0806 14:59:56.706880 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:56.706882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:56.706900 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:56.706968 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:59:56.707263 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.708847 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.710701 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.711441 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.711510 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:56.711615 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:56.714346 (+ 2731us) server_negotiation.cc:299] Negotiation successful
0806 14:59:56.714386 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":916,"thread_start_us":248,"threads_started":1}
I20260806 14:59:56.718351 27662 tablet_service.cc:1467] Processing CreateTablet for tablet 8406062c6032491ba6005fa80d0464ce (DEFAULT_TABLE table=test-table [id=4b760440caa84e6983181c4defcd8e61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:59:56.718782 27662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8406062c6032491ba6005fa80d0464ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:59:56.724498 27772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:56.705158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:43667 (local address 127.0.0.1:48608)
0806 14:59:56.712407 (+ 7249us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:56.712417 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:56.712450 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:56.713023 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:56.713024 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:56.713026 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:59:56.713055 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:56.713098 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.714640 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.714651 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:56.715803 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:56.715813 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.724275 (+ 8462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.724276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:56.724346 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:56.724350 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 14:59:56.724402 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 14:59:56.724446 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7236,"thread_start_us":771,"threads_started":1}
I20260806 14:59:56.724957 27769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:56.705376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48608 (local address 127.11.8.66:43667)
0806 14:59:56.706214 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:56.706220 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:56.712506 (+ 6286us) server_negotiation.cc:407] Connection header received
0806 14:59:56.712569 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:56.712572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:56.712587 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:56.712918 (+ 331us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:59:56.713173 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.714243 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.716332 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.717183 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.717253 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:56.717347 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:56.724795 (+ 7448us) server_negotiation.cc:299] Negotiation successful
0806 14:59:56.724850 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":797,"thread_start_us":237,"threads_started":1}
I20260806 14:59:56.725615 27778 tablet_bootstrap.cc:492] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0: Bootstrap starting.
I20260806 14:59:56.727491 27463 tablet_service.cc:1467] Processing CreateTablet for tablet 8406062c6032491ba6005fa80d0464ce (DEFAULT_TABLE table=test-table [id=4b760440caa84e6983181c4defcd8e61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:59:56.727887 27463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8406062c6032491ba6005fa80d0464ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:59:56.731077 27774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:56.706354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:37649 (local address 127.0.0.1:46858)
0806 14:59:56.714569 (+ 8215us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:56.714580 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:56.714717 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:56.715421 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:56.715422 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:56.715424 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:59:56.715585 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:56.715587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.727393 (+ 11806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.727395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:56.729171 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:56.729175 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.730726 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.730727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:56.730843 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:56.730848 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 14:59:56.730903 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 14:59:56.730964 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8134,"thread_start_us":211,"threads_started":1}
I20260806 14:59:56.731477 27775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:56.706716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46858 (local address 127.11.8.65:37649)
0806 14:59:56.714890 (+ 8174us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:56.714896 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:56.714905 (+ 9us) server_negotiation.cc:407] Connection header received
0806 14:59:56.714939 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:56.714941 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:56.714957 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:56.715259 (+ 302us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:59:56.715676 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.716942 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.729326 (+ 12384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.730526 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.730613 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:56.730672 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:56.731299 (+ 627us) server_negotiation.cc:299] Negotiation successful
0806 14:59:56.731376 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8136,"thread_start_us":162,"threads_started":1}
I20260806 14:59:56.733945 27236 tablet_service.cc:1467] Processing CreateTablet for tablet 8406062c6032491ba6005fa80d0464ce (DEFAULT_TABLE table=test-table [id=4b760440caa84e6983181c4defcd8e61]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 14:59:56.734289 27236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8406062c6032491ba6005fa80d0464ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 14:59:56.736176 27778 tablet_bootstrap.cc:654] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0: Neither blocks nor log segments found. Creating new log.
I20260806 14:59:56.736974 27778 log.cc:824] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0: Log is configured to *not* fsync() on all Append() calls
I20260806 14:59:56.738381 27778 tablet_bootstrap.cc:492] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0: No bootstrap required, opened a new log
I20260806 14:59:56.738476 27778 ts_tablet_manager.cc:1397] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.000s
I20260806 14:59:56.742498 27778 raft_consensus.cc:348] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72656028f1c4d1fa81de87b56d926f0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34207 } } peers { permanent_uuid: "2e20df9f557043e7b2fd47a443fec14e" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37649 } } peers { permanent_uuid: "775c6bb658f34326ad6bb2f24ee82b27" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 43667 } }
I20260806 14:59:56.742942 27778 raft_consensus.cc:374] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:59:56.743047 27778 raft_consensus.cc:729] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d72656028f1c4d1fa81de87b56d926f0, State: Initialized, Role: FOLLOWER
I20260806 14:59:56.743333 27778 consensus_queue.cc:260] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72656028f1c4d1fa81de87b56d926f0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34207 } } peers { permanent_uuid: "2e20df9f557043e7b2fd47a443fec14e" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37649 } } peers { permanent_uuid: "775c6bb658f34326ad6bb2f24ee82b27" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 43667 } }
I20260806 14:59:56.744024 27778 ts_tablet_manager.cc:1428] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260806 14:59:56.744215 27747 heartbeater.cc:503] Master 127.11.8.126:36905 was elected leader, sending a full tablet report...
I20260806 14:59:56.745491 27783 tablet_bootstrap.cc:492] T 8406062c6032491ba6005fa80d0464ce P 2e20df9f557043e7b2fd47a443fec14e: Bootstrap starting.
I20260806 14:59:56.747445 27783 tablet_bootstrap.cc:654] T 8406062c6032491ba6005fa80d0464ce P 2e20df9f557043e7b2fd47a443fec14e: Neither blocks nor log segments found. Creating new log.
I20260806 14:59:56.748183 27783 log.cc:824] T 8406062c6032491ba6005fa80d0464ce P 2e20df9f557043e7b2fd47a443fec14e: Log is configured to *not* fsync() on all Append() calls
I20260806 14:59:56.748293 27779 tablet_bootstrap.cc:492] T 8406062c6032491ba6005fa80d0464ce P 775c6bb658f34326ad6bb2f24ee82b27: Bootstrap starting.
I20260806 14:59:56.750787 27783 tablet_bootstrap.cc:492] T 8406062c6032491ba6005fa80d0464ce P 2e20df9f557043e7b2fd47a443fec14e: No bootstrap required, opened a new log
I20260806 14:59:56.750870 27783 ts_tablet_manager.cc:1397] T 8406062c6032491ba6005fa80d0464ce P 2e20df9f557043e7b2fd47a443fec14e: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260806 14:59:56.752961 27784 raft_consensus.cc:482] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 14:59:56.753059 27784 raft_consensus.cc:504] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72656028f1c4d1fa81de87b56d926f0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34207 } } peers { permanent_uuid: "2e20df9f557043e7b2fd47a443fec14e" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37649 } } peers { permanent_uuid: "775c6bb658f34326ad6bb2f24ee82b27" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 43667 } }
I20260806 14:59:56.753494 27779 tablet_bootstrap.cc:654] T 8406062c6032491ba6005fa80d0464ce P 775c6bb658f34326ad6bb2f24ee82b27: Neither blocks nor log segments found. Creating new log.
I20260806 14:59:56.754274 27784 leader_election.cc:288] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e20df9f557043e7b2fd47a443fec14e (127.11.8.65:37649), 775c6bb658f34326ad6bb2f24ee82b27 (127.11.8.66:43667)
I20260806 14:59:56.756407 27783 raft_consensus.cc:348] T 8406062c6032491ba6005fa80d0464ce P 2e20df9f557043e7b2fd47a443fec14e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72656028f1c4d1fa81de87b56d926f0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34207 } } peers { permanent_uuid: "2e20df9f557043e7b2fd47a443fec14e" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37649 } } peers { permanent_uuid: "775c6bb658f34326ad6bb2f24ee82b27" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 43667 } }
I20260806 14:59:56.756704 27783 raft_consensus.cc:374] T 8406062c6032491ba6005fa80d0464ce P 2e20df9f557043e7b2fd47a443fec14e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:59:56.756749 27783 raft_consensus.cc:729] T 8406062c6032491ba6005fa80d0464ce P 2e20df9f557043e7b2fd47a443fec14e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e20df9f557043e7b2fd47a443fec14e, State: Initialized, Role: FOLLOWER
I20260806 14:59:56.756896 27783 consensus_queue.cc:260] T 8406062c6032491ba6005fa80d0464ce P 2e20df9f557043e7b2fd47a443fec14e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72656028f1c4d1fa81de87b56d926f0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34207 } } peers { permanent_uuid: "2e20df9f557043e7b2fd47a443fec14e" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37649 } } peers { permanent_uuid: "775c6bb658f34326ad6bb2f24ee82b27" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 43667 } }
I20260806 14:59:56.757484 27783 ts_tablet_manager.cc:1428] T 8406062c6032491ba6005fa80d0464ce P 2e20df9f557043e7b2fd47a443fec14e: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260806 14:59:56.757612 27358 heartbeater.cc:503] Master 127.11.8.126:36905 was elected leader, sending a full tablet report...
I20260806 14:59:56.758335 27754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:56.753629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:37649 (local address 127.11.8.67:33673)
0806 14:59:56.753680 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:56.753695 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:56.753755 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:56.754004 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:56.754005 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:56.754007 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:59:56.754193 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:56.754195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.755490 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.755491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:56.756858 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:56.756861 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.758037 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.758039 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:56.758125 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:56.758129 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 14:59:56.758210 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 14:59:56.758260 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 14:59:56.760020 27775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:56.753821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:33673 (local address 127.11.8.65:37649)
0806 14:59:56.753872 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:56.753874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:56.753885 (+ 11us) server_negotiation.cc:407] Connection header received
0806 14:59:56.753914 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:56.753915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:56.753928 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:56.753999 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:59:56.754517 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.755375 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.757120 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.757856 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.757936 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:56.757949 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:56.759918 (+ 1969us) server_negotiation.cc:299] Negotiation successful
0806 14:59:56.759964 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 14:59:56.760298 27285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8406062c6032491ba6005fa80d0464ce" candidate_uuid: "d72656028f1c4d1fa81de87b56d926f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e20df9f557043e7b2fd47a443fec14e" is_pre_election: true
I20260806 14:59:56.760407 27285 raft_consensus.cc:2454] T 8406062c6032491ba6005fa80d0464ce P 2e20df9f557043e7b2fd47a443fec14e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d72656028f1c4d1fa81de87b56d926f0 in term 0.
I20260806 14:59:56.761250 27779 log.cc:824] T 8406062c6032491ba6005fa80d0464ce P 775c6bb658f34326ad6bb2f24ee82b27: Log is configured to *not* fsync() on all Append() calls
I20260806 14:59:56.761412 27595 leader_election.cc:302] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e20df9f557043e7b2fd47a443fec14e, d72656028f1c4d1fa81de87b56d926f0; no voters:
I20260806 14:59:56.761584 27784 raft_consensus.cc:2781] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 14:59:56.761650 27784 raft_consensus.cc:482] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 14:59:56.761688 27784 raft_consensus.cc:3037] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:59:56.763288 27784 raft_consensus.cc:504] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72656028f1c4d1fa81de87b56d926f0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34207 } } peers { permanent_uuid: "2e20df9f557043e7b2fd47a443fec14e" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37649 } } peers { permanent_uuid: "775c6bb658f34326ad6bb2f24ee82b27" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 43667 } }
I20260806 14:59:56.763444 27784 leader_election.cc:288] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0 [CANDIDATE]: Term 1 election: Requested vote from peers 2e20df9f557043e7b2fd47a443fec14e (127.11.8.65:37649), 775c6bb658f34326ad6bb2f24ee82b27 (127.11.8.66:43667)
I20260806 14:59:56.763784 27285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8406062c6032491ba6005fa80d0464ce" candidate_uuid: "d72656028f1c4d1fa81de87b56d926f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e20df9f557043e7b2fd47a443fec14e"
I20260806 14:59:56.763875 27285 raft_consensus.cc:3037] T 8406062c6032491ba6005fa80d0464ce P 2e20df9f557043e7b2fd47a443fec14e [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:59:56.765326 27285 raft_consensus.cc:2454] T 8406062c6032491ba6005fa80d0464ce P 2e20df9f557043e7b2fd47a443fec14e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d72656028f1c4d1fa81de87b56d926f0 in term 1.
I20260806 14:59:56.765592 27595 leader_election.cc:302] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e20df9f557043e7b2fd47a443fec14e, d72656028f1c4d1fa81de87b56d926f0; no voters:
I20260806 14:59:56.766670 27784 raft_consensus.cc:2781] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 14:59:56.767176 27784 raft_consensus.cc:686] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0 [term 1 LEADER]: Becoming Leader. State: Replica: d72656028f1c4d1fa81de87b56d926f0, State: Running, Role: LEADER
I20260806 14:59:56.767293 27784 consensus_queue.cc:237] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72656028f1c4d1fa81de87b56d926f0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34207 } } peers { permanent_uuid: "2e20df9f557043e7b2fd47a443fec14e" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37649 } } peers { permanent_uuid: "775c6bb658f34326ad6bb2f24ee82b27" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 43667 } }
I20260806 14:59:56.768362 27049 catalog_manager.cc:5495] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d72656028f1c4d1fa81de87b56d926f0 (127.11.8.67). New cstate: current_term: 1 leader_uuid: "d72656028f1c4d1fa81de87b56d926f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72656028f1c4d1fa81de87b56d926f0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e20df9f557043e7b2fd47a443fec14e" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "775c6bb658f34326ad6bb2f24ee82b27" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 43667 } health_report { overall_health: UNKNOWN } } }
I20260806 14:59:56.769879 27754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:56.754009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:43667 (local address 127.11.8.67:55035)
0806 14:59:56.758441 (+ 4432us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:56.758455 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 14:59:56.758509 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 14:59:56.758938 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 14:59:56.758940 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 14:59:56.758942 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 14:59:56.758978 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:56.758979 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.759909 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.759910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:56.761107 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 14:59:56.761111 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.769620 (+ 8509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.769622 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 14:59:56.769695 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:56.769700 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 14:59:56.769760 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 14:59:56.769804 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4393,"thread_start_us":232,"threads_started":1}
I20260806 14:59:56.770256 27769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:56.754349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:55035 (local address 127.11.8.66:43667)
0806 14:59:56.754449 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:56.754451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:56.758834 (+ 4383us) server_negotiation.cc:407] Connection header received
0806 14:59:56.758864 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:56.758865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:56.758875 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:56.758930 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 14:59:56.759155 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.759816 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.765776 (+ 5960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.766531 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.766624 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:56.766635 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:56.770134 (+ 3499us) server_negotiation.cc:299] Negotiation successful
0806 14:59:56.770187 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 14:59:56.770637 27494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8406062c6032491ba6005fa80d0464ce" candidate_uuid: "d72656028f1c4d1fa81de87b56d926f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "775c6bb658f34326ad6bb2f24ee82b27" is_pre_election: true
I20260806 14:59:56.772423 27493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8406062c6032491ba6005fa80d0464ce" candidate_uuid: "d72656028f1c4d1fa81de87b56d926f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "775c6bb658f34326ad6bb2f24ee82b27"
W20260806 14:59:56.773233 27596 leader_election.cc:341] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 775c6bb658f34326ad6bb2f24ee82b27 (127.11.8.66:43667): Illegal state: must be running to vote when last-logged opid is not known
W20260806 14:59:56.773386 27596 leader_election.cc:341] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 775c6bb658f34326ad6bb2f24ee82b27 (127.11.8.66:43667): Illegal state: must be running to vote when last-logged opid is not known
I20260806 14:59:56.777127 27779 tablet_bootstrap.cc:492] T 8406062c6032491ba6005fa80d0464ce P 775c6bb658f34326ad6bb2f24ee82b27: No bootstrap required, opened a new log
I20260806 14:59:56.777215 27779 ts_tablet_manager.cc:1397] T 8406062c6032491ba6005fa80d0464ce P 775c6bb658f34326ad6bb2f24ee82b27: Time spent bootstrapping tablet: real 0.030s user 0.000s sys 0.004s
I20260806 14:59:56.779834 27779 raft_consensus.cc:348] T 8406062c6032491ba6005fa80d0464ce P 775c6bb658f34326ad6bb2f24ee82b27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72656028f1c4d1fa81de87b56d926f0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34207 } } peers { permanent_uuid: "2e20df9f557043e7b2fd47a443fec14e" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37649 } } peers { permanent_uuid: "775c6bb658f34326ad6bb2f24ee82b27" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 43667 } }
I20260806 14:59:56.779991 27779 raft_consensus.cc:374] T 8406062c6032491ba6005fa80d0464ce P 775c6bb658f34326ad6bb2f24ee82b27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 14:59:56.780027 27779 raft_consensus.cc:729] T 8406062c6032491ba6005fa80d0464ce P 775c6bb658f34326ad6bb2f24ee82b27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 775c6bb658f34326ad6bb2f24ee82b27, State: Initialized, Role: FOLLOWER
I20260806 14:59:56.780164 27779 consensus_queue.cc:260] T 8406062c6032491ba6005fa80d0464ce P 775c6bb658f34326ad6bb2f24ee82b27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d72656028f1c4d1fa81de87b56d926f0" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 34207 } } peers { permanent_uuid: "2e20df9f557043e7b2fd47a443fec14e" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37649 } } peers { permanent_uuid: "775c6bb658f34326ad6bb2f24ee82b27" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 43667 } }
I20260806 14:59:56.780650 27779 ts_tablet_manager.cc:1428] T 8406062c6032491ba6005fa80d0464ce P 775c6bb658f34326ad6bb2f24ee82b27: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 14:59:56.780790 27559 heartbeater.cc:503] Master 127.11.8.126:36905 was elected leader, sending a full tablet report...
I20260806 14:59:56.786640 27770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:56.782182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58526 (local address 127.11.8.67:34207)
0806 14:59:56.782387 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 14:59:56.782389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 14:59:56.782398 (+ 9us) server_negotiation.cc:407] Connection header received
0806 14:59:56.782483 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 14:59:56.782484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 14:59:56.782508 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 14:59:56.782655 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 14:59:56.783039 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.783884 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.785008 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 14:59:56.785414 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 14:59:56.785487 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 14:59:56.785739 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 14:59:56.785869 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 14:59:56.785947 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 14:59:56.786378 (+ 431us) server_negotiation.cc:299] Negotiation successful
0806 14:59:56.786403 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181}
I20260806 14:59:56.789294 27493 raft_consensus.cc:3037] T 8406062c6032491ba6005fa80d0464ce P 775c6bb658f34326ad6bb2f24ee82b27 [term 0 FOLLOWER]: Advancing to term 1
I20260806 14:59:56.791255 27493 raft_consensus.cc:1261] T 8406062c6032491ba6005fa80d0464ce P 775c6bb658f34326ad6bb2f24ee82b27 [term 1 FOLLOWER]: Refusing update from remote peer d72656028f1c4d1fa81de87b56d926f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:59:56.791745 27793 consensus_queue.cc:1035] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "775c6bb658f34326ad6bb2f24ee82b27" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 43667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:59:56.794180 27285 raft_consensus.cc:1261] T 8406062c6032491ba6005fa80d0464ce P 2e20df9f557043e7b2fd47a443fec14e [term 1 FOLLOWER]: Refusing update from remote peer d72656028f1c4d1fa81de87b56d926f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 14:59:56.794651 27793 consensus_queue.cc:1035] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e20df9f557043e7b2fd47a443fec14e" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 14:59:56.799664 27797 mvcc.cc:205] Tried to move back new op lower bound from 7315572313245794304 to 7315572313159532544. Current Snapshot: MvccSnapshot[applied={T|T < 7315572313245794304}]
I20260806 14:59:56.800962 27802 mvcc.cc:205] Tried to move back new op lower bound from 7315572313245794304 to 7315572313159532544. Current Snapshot: MvccSnapshot[applied={T|T < 7315572313245794304}]
I20260806 14:59:56.801487 27799 mvcc.cc:205] Tried to move back new op lower bound from 7315572313245794304 to 7315572313159532544. Current Snapshot: MvccSnapshot[applied={T|T < 7315572313245794304}]
I20260806 14:59:56.809897 27048 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:59:56.810016 27048 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 14:59:56.811029 27048 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1ede0c91c35f4672bec06d01655036ad: Sending DeleteTablet for 3 replicas of tablet 8406062c6032491ba6005fa80d0464ce
I20260806 14:59:56.811718 27662 tablet_service.cc:1514] Processing DeleteTablet for tablet 8406062c6032491ba6005fa80d0464ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:59:56 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:58524
I20260806 14:59:56.812103 27236 tablet_service.cc:1514] Processing DeleteTablet for tablet 8406062c6032491ba6005fa80d0464ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:59:56 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:46858
I20260806 14:59:56.812798 27463 tablet_service.cc:1514] Processing DeleteTablet for tablet 8406062c6032491ba6005fa80d0464ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 14:59:56 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:48608
I20260806 14:59:56.813529 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 27167
I20260806 14:59:56.830534 27811 tablet_replica.cc:330] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0: stopping tablet replica
I20260806 14:59:56.830781 27811 raft_consensus.cc:2229] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0 [term 1 LEADER]: Raft consensus shutting down.
I20260806 14:59:56.831043 27811 raft_consensus.cc:2258] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:59:56.831770 27811 ts_tablet_manager.cc:1905] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:59:56.838142 27813 tablet_replica.cc:330] T 8406062c6032491ba6005fa80d0464ce P 775c6bb658f34326ad6bb2f24ee82b27: stopping tablet replica
I20260806 14:59:56.838349 27813 raft_consensus.cc:2229] T 8406062c6032491ba6005fa80d0464ce P 775c6bb658f34326ad6bb2f24ee82b27 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 14:59:56.838564 27813 raft_consensus.cc:2258] T 8406062c6032491ba6005fa80d0464ce P 775c6bb658f34326ad6bb2f24ee82b27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 14:59:56.838902 27811 ts_tablet_manager.cc:1918] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 14:59:56.839376 27813 ts_tablet_manager.cc:1905] T 8406062c6032491ba6005fa80d0464ce P 775c6bb658f34326ad6bb2f24ee82b27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 14:59:56.839510 27811 log.cc:1196] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-2/wal/wals/8406062c6032491ba6005fa80d0464ce
I20260806 14:59:56.839893 27811 ts_tablet_manager.cc:1939] T 8406062c6032491ba6005fa80d0464ce P d72656028f1c4d1fa81de87b56d926f0: Deleting consensus metadata
I20260806 14:59:56.841825 27031 catalog_manager.cc:4841] TS d72656028f1c4d1fa81de87b56d926f0 (127.11.8.67:34207): tablet 8406062c6032491ba6005fa80d0464ce (table test-table [id=4b760440caa84e6983181c4defcd8e61]) successfully deleted
I20260806 14:59:56.850677 27813 ts_tablet_manager.cc:1918] T 8406062c6032491ba6005fa80d0464ce P 775c6bb658f34326ad6bb2f24ee82b27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 14:59:56.850898 27813 log.cc:1196] T 8406062c6032491ba6005fa80d0464ce P 775c6bb658f34326ad6bb2f24ee82b27: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786028287744255-11297-0/minicluster-data/ts-1/wal/wals/8406062c6032491ba6005fa80d0464ce
I20260806 14:59:56.851280 27813 ts_tablet_manager.cc:1939] T 8406062c6032491ba6005fa80d0464ce P 775c6bb658f34326ad6bb2f24ee82b27: Deleting consensus metadata
I20260806 14:59:56.851866 27032 catalog_manager.cc:4841] TS 775c6bb658f34326ad6bb2f24ee82b27 (127.11.8.66:43667): tablet 8406062c6032491ba6005fa80d0464ce (table test-table [id=4b760440caa84e6983181c4defcd8e61]) successfully deleted
W20260806 14:59:56.853542 27031 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.65:37649 (error 108)
I20260806 14:59:56.854333 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 27364
I20260806 14:59:56.863772 27774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 14:59:56.863599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:37649 (local address 127.0.0.1:46872)
0806 14:59:56.863664 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0806 14:59:56.863725 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.65:37649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 14:59:56.863932 27031 catalog_manager.cc:4584] TS 2e20df9f557043e7b2fd47a443fec14e (127.11.8.65:37649): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8406062c6032491ba6005fa80d0464ce: Network error: Client connection negotiation failed: client connection to 127.11.8.65:37649: connect: Connection refused (error 111)
I20260806 14:59:56.885819 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 27569
I20260806 14:59:56.893402 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 27006
I20260806 14:59:56.906337 11297 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 14:59:57.250556 11297 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 14:59:57.250690 11297 mini_ranger.cc:66] Stopping Ranger...
I20260806 14:59:57.594679 11297 mini_ranger.cc:68] Stopped Ranger
2026-08-06 14:59:57.594 UTC [26415] LOG: received smart shutdown request
2026-08-06 14:59:57.598 UTC [26415] LOG: background worker "logical replication launcher" (PID 26429) exited with exit code 1
2026-08-06 14:59:57.599 UTC [26419] LOG: shutting down
2026-08-06 14:59:57.764 UTC [26415] LOG: database system is shut down
2026-08-06T14:59:57Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48970 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 14:59:57 dist-test-slave-68xk krb5kdc[27825](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 14:59:57 dist-test-slave-68xk krb5kdc[27825](info): set up 2 sockets
Aug 06 14:59:57 dist-test-slave-68xk krb5kdc[27825](info): commencing operation
krb5kdc: starting...
W20260806 14:59:59.145967 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 14:59:59 dist-test-slave-68xk krb5kdc[27825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T14:59:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T14:59:59Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.11.8.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.11.8.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.11.8.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/rangeradmin_127.11.8.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.11.8.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/rangeradmin_127.11.8.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.11.8.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.11.8.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.11.8.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/rangerlookup_127.11.8.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.11.8.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/rangerlookup_127.11.8.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.8.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.11.8.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.8.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/HTTP_127.11.8.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.8.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/HTTP_127.11.8.84@KRBTEST.COM.keytab.
I20260806 14:59:59.329002 11297 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskF1cjFt/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/postgres -l logfile start
2026-08-06 15:00:01.997 UTC [27859] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 15:00:01.998 UTC [27859] LOG: listening on IPv4 address "127.11.8.84", port 55977
2026-08-06 15:00:02.001 UTC [27859] LOG: listening on Unix socket "/tmp/.s.PGSQL.55977"
2026-08-06 15:00:02.024 UTC [27864] LOG: database system was shut down at 2026-08-06 15:00:00 UTC
2026-08-06 15:00:02.028 UTC [27859] LOG: database system is ready to accept connections
I20260806 15:00:03.223214 11297 mini_postgres.cc:96] Postgres bound to 55977
2026-08-06 15:00:03.230 UTC [27872] FATAL: database "slave" does not exist
127.11.8.84:55977 - accepting connections
I20260806 15:00:03.231386 11297 mini_ranger.cc:161] Starting Ranger...
I20260806 15:00:03.247061 11297 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 15:00:03.740765 11297 mini_ranger.cc:87] Created ranger Postgres database
I20260806 15:00:03.740857 11297 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ranger-admin
2026-08-06 15:00:04,017 [I] DB FLAVOR :POSTGRES
2026-08-06 15:00:04,018 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 15:00:04,018 [I] Checking connection..
2026-08-06 15:00:04,018 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:55977/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 15:00:04,287 [I] Checking connection passed.
2026-08-06 15:00:04,288 [I] --------- Verifying version history table ---------
2026-08-06 15:00:04,288 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:55977/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 15:00:04,574 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 15:00:04,574 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:55977/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 15:00:04,825 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 15:00:04,826 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 15:00:04,826 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:55977/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 15:00:05.072 UTC [27963] WARNING: there is no transaction in progress
2026-08-06 15:00:05,081 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 15:00:05,082 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:55977/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 15:00:05,333 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 15:00:05,333 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 15:00:05,334 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:55977/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 15:00:05,583 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:55977/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 15:00:05,848 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:55977/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-68xk.c.gcp-upstream.internal','N') ;"
2026-08-06 15:00:06,113 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:55977/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 15:00:06,360 [I] Table x_portal_user does not exist in database ranger
2026-08-06 15:00:06,360 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:55977/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 15:00:06,596 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 15:00:06,597 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:55977/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 15:00:06,850 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:55977/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 15:00:07,119 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 15:00:07,119 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:55977/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 15:00:08.095 UTC [28117] WARNING: there is no transaction in progress
2026-08-06 15:00:08.112 UTC [28117] WARNING: there is no transaction in progress
2026-08-06 15:00:08.129 UTC [28117] WARNING: there is no transaction in progress
2026-08-06 15:00:08.145 UTC [28117] WARNING: there is no transaction in progress
2026-08-06 15:00:08.161 UTC [28117] WARNING: there is no transaction in progress
2026-08-06 15:00:08.181 UTC [28117] WARNING: there is no transaction in progress
2026-08-06 15:00:08.248 UTC [28117] WARNING: there is no transaction in progress
2026-08-06 15:00:08.255 UTC [28117] WARNING: there is no transaction in progress
2026-08-06 15:00:08.269 UTC [28117] WARNING: there is no transaction in progress
2026-08-06 15:00:08.279 UTC [28117] WARNING: there is no transaction in progress
2026-08-06 15:00:08.289 UTC [28117] WARNING: there is no transaction in progress
2026-08-06 15:00:08.303 UTC [28117] WARNING: there is no transaction in progress
2026-08-06 15:00:08.313 UTC [28117] WARNING: there is no transaction in progress
2026-08-06 15:00:08.320 UTC [28117] WARNING: there is no transaction in progress
2026-08-06 15:00:08.327 UTC [28117] WARNING: there is no transaction in progress
2026-08-06 15:00:08.334 UTC [28117] WARNING: there is no transaction in progress
2026-08-06 15:00:08,993 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 15:00:08,994 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:55977/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 15:00:09,258 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 15:00:09,258 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:55977/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 15:00:09,512 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 15:00:09,512 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:55977/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 15:00:09,782 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 15:00:09,782 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:55977/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 15:00:10,058 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:55977/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 15:00:10,293 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:55977/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-68xk.c.gcp-upstream.internal';"
2026-08-06 15:00:10,547 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 15:00:10,547 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:55977/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 15:00:10,777 [I] DB_PATCHES have already been applied
I20260806 15:00:10.783397 11297 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ranger-admin:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskF1cjFt/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 15:00:10.783464 11297 mini_ranger.cc:193] Using host: 127.11.8.84
I20260806 15:00:10.787386 11297 mini_ranger.cc:227] Ranger admin URL: http://127.11.8.84:50945
Aug 06, 2026 3:00:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 3:00:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 3:00:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 3:00:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 3:00:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 3:00:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 3:00:11 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 15:00:12.110200 11297 mini_ranger.cc:160] Time spent starting Ranger: real 8.879s user 0.000s sys 0.010s
Aug 06, 2026 3:00:33 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 3:00:33 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 3:00:33 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 3:00:33 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 3:00:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 3:00:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 3:00:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 3:00:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 3:00:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 3:00:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 3:00:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 3:00:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 3:00:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 3:00:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 3:00:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 3:00:33 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 3:00:33 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 3:00:34 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 15:00:34.717695 11297 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.11.8.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.11.8.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.11.8.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/rangerkms_127.11.8.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.11.8.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/rangerkms_127.11.8.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.11.8.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.11.8.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.11.8.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/HTTP_127.11.8.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.11.8.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/HTTP_127.11.8.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 15:00:34 dist-test-slave-68xk krb5kdc[27825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028434, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 15:00:34.816150 11297 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 15:00:34.826213 11297 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 15:00:35.716825 11297 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 15:00:35.716914 11297 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ranger-kms
I20260806 15:00:35.716933 11297 mini_ranger_kms.cc:227] Using postgres at 127.11.8.84:55977
2026-08-06 15:00:35,896 [I] DB FLAVOR :POSTGRES
2026-08-06 15:00:35,896 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 15:00:35,896 [I] Checking connection
2026-08-06 15:00:35,896 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:55977/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 15:00:36,155 [I] connection success
2026-08-06 15:00:36,156 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 15:00:36,156 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 15:00:36,156 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:55977/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 15:00:36,400 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 15:00:36,400 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 15:00:36,400 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 15:00:36,400 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.11.8.84:55977/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 15:00:36,679 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 15:00:37.190326 11297 mini_ranger_kms.cc:310] Created kms service
I20260806 15:00:37.382277 11297 mini_ranger_kms.cc:326] Created kudu user
I20260806 15:00:37.424053 11297 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 15:00:37 dist-test-slave-68xk krb5kdc[27825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028434, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.11.8.84@KRBTEST.COM
I20260806 15:00:37.842819 11297 mini_ranger_kms.cc:384] Added ranger policy
I20260806 15:00:37.843070 11297 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ranger-kms:/tmp/dist-test-taskF1cjFt/build/release/bin/postgresql.jar:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskF1cjFt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 15:00:37.843098 11297 mini_ranger_kms.cc:240] Using host: 127.11.8.84
I20260806 15:00:37.848037 11297 mini_ranger_kms.cc:276] Ranger KMS PID: 28378
I20260806 15:00:37.848124 11297 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.11.8.84:59751
Aug 06, 2026 3:00:38 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 3:00:38 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 3:00:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 3:00:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 3:00:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 3:00:39 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 15:00:39 dist-test-slave-68xk krb5kdc[27825](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786028439, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.8.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 15:00:40 dist-test-slave-68xk krb5kdc[27825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028439, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.8.84@KRBTEST.COM for HTTP/127.11.8.84@KRBTEST.COM
I20260806 15:00:40.641268 11297 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.825s user 0.010s sys 0.152s
I20260806 15:00:40.641382 11297 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 15:00:40.641431 11297 mini_ranger_kms.cc:401] 127.11.8.84:59751/kms/v1/keys
Aug 06 15:00:40 dist-test-slave-68xk krb5kdc[27825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028439, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.11.8.84@KRBTEST.COM for HTTP/127.11.8.84@KRBTEST.COM
Aug 06 15:00:43 dist-test-slave-68xk krb5kdc[27825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 15:00:43.061277 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:44625
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:36175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:44625
--ranger_config_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.8.84:59751/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:00:43.147208 28447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:00:43.147441 28447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:00:43.147477 28447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:00:43.148627 28447 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:00:43.148691 28447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:00:43.148715 28447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:00:43.148737 28447 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:00:43.148765 28447 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:00:43.150314 28447 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:36175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/wal
--ranger_kms_url=127.11.8.84:59751/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:44625
--ranger_config_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:44625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:00:43.150616 28447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:00:43.150857 28447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:00:43.156162 28455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:43.156378 28447 server_base.cc:1034] running on GCE node
W20260806 15:00:43.156548 28454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:43.156594 28458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:43.157083 28447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:00:43.157465 28447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:00:43.158639 28447 hybrid_clock.cc:648] HybridClock initialized: now 1786028443158587 us; error 65 us; skew 500 ppm
Aug 06 15:00:43 dist-test-slave-68xk krb5kdc[27825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028443, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:00:43.168179 28447 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 15:00:43.168779 28447 webserver.cc:460] Webserver started at http://127.11.8.126:42843/ using document root <none> and password file <none>
I20260806 15:00:43.169090 28447 fs_manager.cc:359] Metadata directory not provided
I20260806 15:00:43.169155 28447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:00:43.169302 28447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 15:00:43 dist-test-slave-68xk krb5kdc[27825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028443, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for HTTP/127.11.8.84@KRBTEST.COM
I20260806 15:00:43.222020 28447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "409e8d7db85240958e76dda03cc6675e"
format_stamp: "Formatted at 2026-08-06 15:00:43 on dist-test-slave-68xk"
server_key: "bef36a0de5f0e42a6740fe97495b5b4d"
server_key_iv: "c8b4f04543e68f606f22c05d35a6be41"
server_key_version: "kuduclusterkey@0"
I20260806 15:00:43.222452 28447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "409e8d7db85240958e76dda03cc6675e"
format_stamp: "Formatted at 2026-08-06 15:00:43 on dist-test-slave-68xk"
server_key: "bef36a0de5f0e42a6740fe97495b5b4d"
server_key_iv: "c8b4f04543e68f606f22c05d35a6be41"
server_key_version: "kuduclusterkey@0"
I20260806 15:00:43.223945 28447 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:00:43.235212 28467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:43.235399 28447 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 15:00:43.235502 28447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "409e8d7db85240958e76dda03cc6675e"
format_stamp: "Formatted at 2026-08-06 15:00:43 on dist-test-slave-68xk"
server_key: "bef36a0de5f0e42a6740fe97495b5b4d"
server_key_iv: "c8b4f04543e68f606f22c05d35a6be41"
server_key_version: "kuduclusterkey@0"
I20260806 15:00:43.235615 28447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:00:43.250646 28447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:00:43.252712 28447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:00:43.252900 28447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:00:43.257879 28447 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:44625
I20260806 15:00:43.257915 28545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:44625 every 8 connection(s)
I20260806 15:00:43.258335 28447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:00:43.259194 28546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:00:43.262212 28546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e: Bootstrap starting.
I20260806 15:00:43.263008 28546 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e: Neither blocks nor log segments found. Creating new log.
I20260806 15:00:43.263332 28546 log.cc:824] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e: Log is configured to *not* fsync() on all Append() calls
I20260806 15:00:43.264153 28546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e: No bootstrap required, opened a new log
I20260806 15:00:43.267012 28546 raft_consensus.cc:348] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409e8d7db85240958e76dda03cc6675e" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44625 } }
I20260806 15:00:43.267201 28546 raft_consensus.cc:374] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:00:43.267236 28546 raft_consensus.cc:729] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 409e8d7db85240958e76dda03cc6675e, State: Initialized, Role: FOLLOWER
I20260806 15:00:43.267377 28546 consensus_queue.cc:260] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409e8d7db85240958e76dda03cc6675e" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44625 } }
I20260806 15:00:43.267449 28546 raft_consensus.cc:388] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:00:43.267479 28546 raft_consensus.cc:482] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:00:43.267524 28546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:00:43.268198 28546 raft_consensus.cc:504] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409e8d7db85240958e76dda03cc6675e" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44625 } }
I20260806 15:00:43.268303 28546 leader_election.cc:302] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 409e8d7db85240958e76dda03cc6675e; no voters:
I20260806 15:00:43.268409 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 28447
I20260806 15:00:43.268457 28546 leader_election.cc:288] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:00:43.268525 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:00:43.268524 28550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:00:43.268694 28546 sys_catalog.cc:564] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:00:43.268683 28550 raft_consensus.cc:686] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [term 1 LEADER]: Becoming Leader. State: Replica: 409e8d7db85240958e76dda03cc6675e, State: Running, Role: LEADER
I20260806 15:00:43.268738 11297 external_mini_cluster.cc:1428] Setting key 94d94027cfdace004d6ad4bd63717167
I20260806 15:00:43.268822 28550 consensus_queue.cc:237] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409e8d7db85240958e76dda03cc6675e" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44625 } }
I20260806 15:00:43.269134 28550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "409e8d7db85240958e76dda03cc6675e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409e8d7db85240958e76dda03cc6675e" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44625 } } }
I20260806 15:00:43.269207 28550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [sys.catalog]: This master's current role is: LEADER
I20260806 15:00:43.269336 28550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 409e8d7db85240958e76dda03cc6675e. Latest consensus state: current_term: 1 leader_uuid: "409e8d7db85240958e76dda03cc6675e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409e8d7db85240958e76dda03cc6675e" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44625 } } }
I20260806 15:00:43.269387 28550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [sys.catalog]: This master's current role is: LEADER
I20260806 15:00:43.270125 28561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:00:43.270195 28546 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 15:00:43.273888 28561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 15:00:43 dist-test-slave-68xk krb5kdc[27825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:00:43.284313 28565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:43.269418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51444 (local address 127.11.8.126:44625)
0806 15:00:43.269588 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:43.269597 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:43.269903 (+ 306us) server_negotiation.cc:407] Connection header received
0806 15:00:43.270143 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:43.270150 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:43.270211 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:43.270355 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:43.272502 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:43.275479 (+ 2977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:43.276360 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:43.276860 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:43.277014 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:43.279446 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:43.279450 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:43.279450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:43.279460 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:43.281957 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:43.282341 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:43.282344 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:43.282345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:43.282535 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:43.282814 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:43.282815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:43.282816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:43.283214 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:43.283779 (+ 565us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:43.283866 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 15:00:43.284001 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":72,"threads_started":1}
I20260806 15:00:43.290823 28561 catalog_manager.cc:1347] Generated new cluster ID: ebd9ac309f93465b8ce3869eac42db29
I20260806 15:00:43.290894 28561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:00:43.300189 28561 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:00:43.300928 28561 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:00:43.310951 28561 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e: Generated new TSK 0
I20260806 15:00:43.311141 28561 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
I20260806 15:00:44.050179 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.8.84:59751/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:0
--local_ip_for_outbound_sockets=127.11.8.65
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:44625
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:36175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:00:44.173465 28606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:00:44.173779 28606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:00:44.173817 28606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:00:44.175590 28606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:00:44.175660 28606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:00:44.175704 28606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 15:00:44.177886 28606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:36175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.11.8.84:59751/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:44625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:00:44.178189 28606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:00:44.178519 28606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:00:44.184170 28615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:44.184275 28619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:44.184175 28614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:44.184567 28606 server_base.cc:1034] running on GCE node
I20260806 15:00:44.184787 28606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:00:44.185060 28606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:00:44.186245 28606 hybrid_clock.cc:648] HybridClock initialized: now 1786028444186229 us; error 33 us; skew 500 ppm
Aug 06 15:00:44 dist-test-slave-68xk krb5kdc[27825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028444, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:00:44.189227 28606 init.cc:377] Logged in from keytab as kudu/127.11.8.65@KRBTEST.COM (short username kudu)
I20260806 15:00:44.189839 28606 webserver.cc:460] Webserver started at http://127.11.8.65:34661/ using document root <none> and password file <none>
I20260806 15:00:44.190114 28606 fs_manager.cc:359] Metadata directory not provided
I20260806 15:00:44.190182 28606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:00:44.190318 28606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 15:00:44 dist-test-slave-68xk krb5kdc[27825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028444, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for HTTP/127.11.8.84@KRBTEST.COM
I20260806 15:00:44.205374 28606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/data/instance:
uuid: "898ae30bbbf34e5688ac81b6ee8bf3c1"
format_stamp: "Formatted at 2026-08-06 15:00:44 on dist-test-slave-68xk"
server_key: "71cbc366b1428dffc23278e73bf40f75"
server_key_iv: "697d439eb23f902eac1dbe4750a24e09"
server_key_version: "kuduclusterkey@0"
I20260806 15:00:44.205947 28606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance:
uuid: "898ae30bbbf34e5688ac81b6ee8bf3c1"
format_stamp: "Formatted at 2026-08-06 15:00:44 on dist-test-slave-68xk"
server_key: "71cbc366b1428dffc23278e73bf40f75"
server_key_iv: "697d439eb23f902eac1dbe4750a24e09"
server_key_version: "kuduclusterkey@0"
I20260806 15:00:44.208129 28606 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 15:00:44.222170 28629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:44.222391 28606 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:00:44.222508 28606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "898ae30bbbf34e5688ac81b6ee8bf3c1"
format_stamp: "Formatted at 2026-08-06 15:00:44 on dist-test-slave-68xk"
server_key: "71cbc366b1428dffc23278e73bf40f75"
server_key_iv: "697d439eb23f902eac1dbe4750a24e09"
server_key_version: "kuduclusterkey@0"
I20260806 15:00:44.222607 28606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:00:44.242718 28606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:00:44.246107 28606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:00:44.246272 28606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:00:44.246532 28606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:00:44.246953 28606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:00:44.247037 28606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:44.247100 28606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:00:44.247145 28606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:44.254576 28606 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:43639
I20260806 15:00:44.255087 28606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
I20260806 15:00:44.257788 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 28606
I20260806 15:00:44.257902 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance
I20260806 15:00:44.258126 11297 external_mini_cluster.cc:1428] Setting key 5be1e94c9b68a7d5e81852cd11de255f
I20260806 15:00:44.264871 28794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:43639 every 8 connection(s)
Aug 06 15:00:44 dist-test-slave-68xk krb5kdc[27825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028444, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.8.66@KRBTEST.COM; defaulting to no policy
I20260806 15:00:44.278635 28801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:44.265658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:38251 (local address 127.11.8.126:44625)
0806 15:00:44.266132 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:44.266134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:44.267064 (+ 930us) server_negotiation.cc:407] Connection header received
0806 15:00:44.267094 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:44.267095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:44.267108 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:44.267171 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:44.267939 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:44.268734 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:44.270265 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:44.270606 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:44.270649 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:44.273416 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:44.273418 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:44.273418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:44.273427 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:44.275579 (+ 2152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:44.276107 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:44.276108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:44.276109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:44.276176 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:44.276465 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:44.276466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:44.276466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:44.276604 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:44.276674 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:44.278321 (+ 1647us) server_negotiation.cc:299] Negotiation successful
0806 15:00:44.278480 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":95,"threads_started":1}
I20260806 15:00:44.279501 28802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:44.265662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:44625 (local address 127.11.8.65:38251)
0806 15:00:44.266191 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:44.266238 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:44.266898 (+ 660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:44.267301 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:44.267306 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:44.267555 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:00:44.267826 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:44.267833 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:44.268855 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:44.268856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:44.270160 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:44.270163 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:44.270758 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:44.270759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:44.270900 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:44.271538 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:00:44.271560 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:00:44.273285 (+ 1725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:00:44.275683 (+ 2398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:44.275690 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:44.275692 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:44.276012 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:44.276257 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:44.276258 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:44.276258 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:44.276387 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:44.276703 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:00:44.276710 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:00:44.276892 (+ 182us) client_negotiation.cc:769] Sending connection context
0806 15:00:44.278251 (+ 1359us) client_negotiation.cc:240] Negotiation successful
0806 15:00:44.279262 (+ 1011us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":81,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.66@KRBTEST.COM" created.
I20260806 15:00:44.280297 28795 heartbeater.cc:347] Connected to a master server at 127.11.8.126:44625
I20260806 15:00:44.280406 28795 heartbeater.cc:464] Registering TS with master...
I20260806 15:00:44.280622 28795 heartbeater.cc:511] Master 127.11.8.126:44625 requested a full tablet report, sending...
I20260806 15:00:44.281304 28489 ts_manager.cc:194] Registered new tserver with Master: 898ae30bbbf34e5688ac81b6ee8bf3c1 (127.11.8.65:43639)
I20260806 15:00:44.282186 28489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.65@KRBTEST.COM'} at 127.11.8.65:38251
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
I20260806 15:00:44.301941 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.8.84:59751/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:0
--local_ip_for_outbound_sockets=127.11.8.66
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:44625
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:36175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 15:00:44 dist-test-slave-68xk krb5kdc[27825](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786028444, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 15:00:44.429417 28805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:00:44.429809 28805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:00:44.429898 28805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:00:44.431885 28805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:00:44.431994 28805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:00:44.432073 28805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 15:00:44.434549 28805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:36175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.11.8.84:59751/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:44625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:00:44.434988 28805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:00:44.435334 28805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:00:44.441946 28813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:44.441946 28816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:44.442277 28812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:44.443526 28805 server_base.cc:1034] running on GCE node
I20260806 15:00:44.443763 28805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:00:44.444051 28805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:00:44.445199 28805 hybrid_clock.cc:648] HybridClock initialized: now 1786028444445176 us; error 37 us; skew 500 ppm
Aug 06 15:00:44 dist-test-slave-68xk krb5kdc[27825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028444, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:00:44.452023 28805 init.cc:377] Logged in from keytab as kudu/127.11.8.66@KRBTEST.COM (short username kudu)
I20260806 15:00:44.452529 28805 webserver.cc:460] Webserver started at http://127.11.8.66:40633/ using document root <none> and password file <none>
I20260806 15:00:44.452797 28805 fs_manager.cc:359] Metadata directory not provided
I20260806 15:00:44.452864 28805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:00:44.453007 28805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 15:00:44 dist-test-slave-68xk krb5kdc[27825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028444, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for HTTP/127.11.8.84@KRBTEST.COM
I20260806 15:00:44.467468 28805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/data/instance:
uuid: "d06a80aa4042458f94b96a1e0d37fd5d"
format_stamp: "Formatted at 2026-08-06 15:00:44 on dist-test-slave-68xk"
server_key: "46f25be8186f42b3f7ea028ff56e9764"
server_key_iv: "6e9baaeff07f5782c781adfbb6a11406"
server_key_version: "kuduclusterkey@0"
I20260806 15:00:44.467958 28805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance:
uuid: "d06a80aa4042458f94b96a1e0d37fd5d"
format_stamp: "Formatted at 2026-08-06 15:00:44 on dist-test-slave-68xk"
server_key: "46f25be8186f42b3f7ea028ff56e9764"
server_key_iv: "6e9baaeff07f5782c781adfbb6a11406"
server_key_version: "kuduclusterkey@0"
I20260806 15:00:44.470431 28805 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260806 15:00:44.490528 28825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:44.490741 28805 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260806 15:00:44.490849 28805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "d06a80aa4042458f94b96a1e0d37fd5d"
format_stamp: "Formatted at 2026-08-06 15:00:44 on dist-test-slave-68xk"
server_key: "46f25be8186f42b3f7ea028ff56e9764"
server_key_iv: "6e9baaeff07f5782c781adfbb6a11406"
server_key_version: "kuduclusterkey@0"
I20260806 15:00:44.490957 28805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:00:44.516578 28805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:00:44.518743 28805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:00:44.518926 28805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:00:44.519263 28805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:00:44.519680 28805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:00:44.519747 28805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:44.519786 28805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:00:44.519812 28805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:44.528199 28805 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:37985
I20260806 15:00:44.528690 28805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
I20260806 15:00:44.529253 28987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:37985 every 8 connection(s)
I20260806 15:00:44.534991 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 28805
I20260806 15:00:44.535120 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance
I20260806 15:00:44.535362 11297 external_mini_cluster.cc:1428] Setting key 6cd871c232456899ddc028a5df44bd4e
Aug 06 15:00:44 dist-test-slave-68xk krb5kdc[27825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028444, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:00:44.541908 28801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:44.530338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:53693 (local address 127.11.8.126:44625)
0806 15:00:44.530391 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:44.530394 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:44.530814 (+ 420us) server_negotiation.cc:407] Connection header received
0806 15:00:44.531511 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:44.531512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:44.531527 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:44.531593 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:44.532400 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:44.533167 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:44.534240 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:44.534603 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:44.534687 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:44.537972 (+ 3285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:44.537975 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:44.537975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:44.537985 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:44.540106 (+ 2121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:44.540631 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:44.540632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:44.540633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:44.540706 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:44.541032 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:44.541034 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:44.541034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:44.541222 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:44.541306 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:44.541683 (+ 377us) server_negotiation.cc:299] Negotiation successful
0806 15:00:44.541789 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 15:00:44.542048 28990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:44.530004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:44625 (local address 127.11.8.66:53693)
0806 15:00:44.530665 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:44.530712 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:44.531384 (+ 672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:44.531667 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:44.531671 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:44.531964 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:00:44.532268 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:44.532275 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:44.533275 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:44.533276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:44.534135 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:44.534138 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:44.535266 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:44.535267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:44.535415 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:44.536021 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:00:44.536032 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:00:44.537856 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:00:44.540212 (+ 2356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:44.540220 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:44.540223 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:44.540534 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:44.540792 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:44.540794 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:44.540794 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:44.540941 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:44.541316 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:00:44.541322 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:00:44.541501 (+ 179us) client_negotiation.cc:769] Sending connection context
0806 15:00:44.541635 (+ 134us) client_negotiation.cc:240] Negotiation successful
0806 15:00:44.541804 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":113,"threads_started":1}
I20260806 15:00:44.542613 28988 heartbeater.cc:347] Connected to a master server at 127.11.8.126:44625
I20260806 15:00:44.542732 28988 heartbeater.cc:464] Registering TS with master...
I20260806 15:00:44.542961 28988 heartbeater.cc:511] Master 127.11.8.126:44625 requested a full tablet report, sending...
I20260806 15:00:44.543453 28489 ts_manager.cc:194] Registered new tserver with Master: d06a80aa4042458f94b96a1e0d37fd5d (127.11.8.66:37985)
I20260806 15:00:44.544052 28489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.66@KRBTEST.COM'} at 127.11.8.66:53693
WARNING: no policy specified for kudu/127.11.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
I20260806 15:00:44.565608 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.8.84:59751/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:0
--local_ip_for_outbound_sockets=127.11.8.67
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:44625
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:36175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:00:44.686645 28993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:00:44.686887 28993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:00:44.686930 28993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:00:44.689013 28993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:00:44.689073 28993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:00:44.689118 28993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 15:00:44.691286 28993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:36175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.11.8.84:59751/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:44625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:00:44.691541 28993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:00:44.691839 28993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:00:44.697521 29002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:44.697530 29001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:44.698004 29004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:44.698251 28993 server_base.cc:1034] running on GCE node
I20260806 15:00:44.698511 28993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:00:44.698786 28993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:00:44.699963 28993 hybrid_clock.cc:648] HybridClock initialized: now 1786028444699945 us; error 36 us; skew 500 ppm
Aug 06 15:00:44 dist-test-slave-68xk krb5kdc[27825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028444, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:00:44.702916 28993 init.cc:377] Logged in from keytab as kudu/127.11.8.67@KRBTEST.COM (short username kudu)
I20260806 15:00:44.703372 28993 webserver.cc:460] Webserver started at http://127.11.8.67:42357/ using document root <none> and password file <none>
I20260806 15:00:44.703652 28993 fs_manager.cc:359] Metadata directory not provided
I20260806 15:00:44.703732 28993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:00:44.703900 28993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 15:00:44 dist-test-slave-68xk krb5kdc[27825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028444, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for HTTP/127.11.8.84@KRBTEST.COM
I20260806 15:00:44.717383 28993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/data/instance:
uuid: "ff01961beab84ed6b6d67c43a77ed5d9"
format_stamp: "Formatted at 2026-08-06 15:00:44 on dist-test-slave-68xk"
server_key: "6fa5d7db443d1f150ade2ee2c271ec01"
server_key_iv: "86894a322c6770d7d4282b60c74cc3df"
server_key_version: "kuduclusterkey@0"
I20260806 15:00:44.717878 28993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance:
uuid: "ff01961beab84ed6b6d67c43a77ed5d9"
format_stamp: "Formatted at 2026-08-06 15:00:44 on dist-test-slave-68xk"
server_key: "6fa5d7db443d1f150ade2ee2c271ec01"
server_key_iv: "86894a322c6770d7d4282b60c74cc3df"
server_key_version: "kuduclusterkey@0"
I20260806 15:00:44.720185 28993 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260806 15:00:44.744465 29012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:44.744661 28993 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.001s sys 0.000s
I20260806 15:00:44.744776 28993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "ff01961beab84ed6b6d67c43a77ed5d9"
format_stamp: "Formatted at 2026-08-06 15:00:44 on dist-test-slave-68xk"
server_key: "6fa5d7db443d1f150ade2ee2c271ec01"
server_key_iv: "86894a322c6770d7d4282b60c74cc3df"
server_key_version: "kuduclusterkey@0"
I20260806 15:00:44.744887 28993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:00:44.772465 28993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:00:44.775144 28993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:00:44.775338 28993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:00:44.775620 28993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:00:44.776017 28993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:00:44.776068 28993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:44.776101 28993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:00:44.776120 28993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:44.785034 28993 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:40563
I20260806 15:00:44.785204 29183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:40563 every 8 connection(s)
I20260806 15:00:44.785513 28993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
I20260806 15:00:44.790657 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 28993
I20260806 15:00:44.790791 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance
I20260806 15:00:44.791003 11297 external_mini_cluster.cc:1428] Setting key 458ffdf16e17353f20f404c8e85bc62b
Aug 06 15:00:44 dist-test-slave-68xk krb5kdc[27825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028444, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:00:44.799026 28801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:44.787241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:59615 (local address 127.11.8.126:44625)
0806 15:00:44.787320 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:44.787322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:44.787579 (+ 257us) server_negotiation.cc:407] Connection header received
0806 15:00:44.788282 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:44.788284 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:44.788298 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:44.788386 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:44.789154 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:44.789906 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:44.792062 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:44.792374 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:44.792423 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:44.795229 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:44.795231 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:44.795231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:44.795240 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:44.797161 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:44.797696 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:44.797698 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:44.797698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:44.797770 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:44.798102 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:44.798103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:44.798103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:44.798261 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:44.798338 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:44.798820 (+ 482us) server_negotiation.cc:299] Negotiation successful
0806 15:00:44.798915 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 15:00:44.799595 29186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:44.787208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:44625 (local address 127.11.8.67:59615)
0806 15:00:44.787439 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:44.787487 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:44.788149 (+ 662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:44.788463 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:44.788468 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:44.788724 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:00:44.789012 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:44.789019 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:44.790043 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:44.790044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:44.791953 (+ 1909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:44.791956 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:44.792533 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:44.792534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:44.792676 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:44.793294 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:00:44.793306 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:00:44.795090 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:00:44.797264 (+ 2174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:44.797272 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:44.797274 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:44.797596 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:44.797857 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:44.797858 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:44.797858 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:44.798007 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:44.798372 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:00:44.798379 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:00:44.798625 (+ 246us) client_negotiation.cc:769] Sending connection context
0806 15:00:44.799137 (+ 512us) client_negotiation.cc:240] Negotiation successful
0806 15:00:44.799319 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":126,"threads_started":1}
I20260806 15:00:44.800194 29184 heartbeater.cc:347] Connected to a master server at 127.11.8.126:44625
I20260806 15:00:44.800304 29184 heartbeater.cc:464] Registering TS with master...
I20260806 15:00:44.800506 29184 heartbeater.cc:511] Master 127.11.8.126:44625 requested a full tablet report, sending...
I20260806 15:00:44.800930 28489 ts_manager.cc:194] Registered new tserver with Master: ff01961beab84ed6b6d67c43a77ed5d9 (127.11.8.67:40563)
I20260806 15:00:44.801441 28489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.67@KRBTEST.COM'} at 127.11.8.67:59615
I20260806 15:00:44.802950 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:00:44.803690 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 28606
I20260806 15:00:44.813259 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 28805
I20260806 15:00:44.841063 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 28993
I20260806 15:00:44.847895 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 28447
I20260806 15:00:44.857779 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:44625
--webserver_interface=127.11.8.126
--webserver_port=42843
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:36175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:44625
--ranger_config_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.8.84:59751/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.8.84:59751/kms with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:00:44.939769 29187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:00:44.939980 29187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:00:44.940003 29187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:00:44.941154 29187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:00:44.941196 29187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:00:44.941211 29187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:00:44.941223 29187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:00:44.941236 29187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:00:44.942948 29187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:36175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.8.84:59751/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:44625
--ranger_config_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:44625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=42843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:00:44.943204 29187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:00:44.943431 29187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:00:44.948283 29192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:44.948302 29196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:44.948365 29194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:44.948431 29187 server_base.cc:1034] running on GCE node
I20260806 15:00:44.948748 29187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:00:44.949047 29187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:00:44.950218 29187 hybrid_clock.cc:648] HybridClock initialized: now 1786028444950197 us; error 39 us; skew 500 ppm
Aug 06 15:00:44 dist-test-slave-68xk krb5kdc[27825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028444, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:00:44.952700 29187 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 15:00:44.953115 29187 webserver.cc:460] Webserver started at http://127.11.8.126:42843/ using document root <none> and password file <none>
I20260806 15:00:44.953356 29187 fs_manager.cc:359] Metadata directory not provided
I20260806 15:00:44.953434 29187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 15:00:44 dist-test-slave-68xk krb5kdc[27825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028444, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for HTTP/127.11.8.84@KRBTEST.COM
I20260806 15:00:45.002792 29187 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260806 15:00:45.003595 29208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:45.003752 29187 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:00:45.003846 29187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "409e8d7db85240958e76dda03cc6675e"
format_stamp: "Formatted at 2026-08-06 15:00:43 on dist-test-slave-68xk"
server_key: "bef36a0de5f0e42a6740fe97495b5b4d"
server_key_iv: "c8b4f04543e68f606f22c05d35a6be41"
server_key_version: "kuduclusterkey@0"
I20260806 15:00:45.004211 29187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:00:45.019550 29187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:00:45.021677 29187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:00:45.021842 29187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:00:45.026232 29187 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:44625
I20260806 15:00:45.026270 29276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:44625 every 8 connection(s)
I20260806 15:00:45.026701 29187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:00:45.027870 29277 sys_catalog.cc:263] Verifying existing consensus state
I20260806 15:00:45.028524 29277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e: Bootstrap starting.
I20260806 15:00:45.034126 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 29187
I20260806 15:00:45.034523 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.8.84:59751/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:43639
--local_ip_for_outbound_sockets=127.11.8.65
--tserver_master_addrs=127.11.8.126:44625
--webserver_port=34661
--webserver_interface=127.11.8.65
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:36175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.8.84:59751/kms with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:00:45.037004 29277 log.cc:824] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e: Log is configured to *not* fsync() on all Append() calls
I20260806 15:00:45.038877 29277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 15:00:45.039176 29277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e: Bootstrap complete.
I20260806 15:00:45.041543 29277 raft_consensus.cc:348] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409e8d7db85240958e76dda03cc6675e" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44625 } }
I20260806 15:00:45.041765 29277 raft_consensus.cc:729] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 409e8d7db85240958e76dda03cc6675e, State: Initialized, Role: FOLLOWER
I20260806 15:00:45.041934 29277 consensus_queue.cc:260] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409e8d7db85240958e76dda03cc6675e" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44625 } }
I20260806 15:00:45.042006 29277 raft_consensus.cc:388] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:00:45.042034 29277 raft_consensus.cc:482] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:00:45.042099 29277 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [term 1 FOLLOWER]: Advancing to term 2
I20260806 15:00:45.043207 29277 raft_consensus.cc:504] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409e8d7db85240958e76dda03cc6675e" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44625 } }
I20260806 15:00:45.043304 29277 leader_election.cc:302] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 409e8d7db85240958e76dda03cc6675e; no voters:
I20260806 15:00:45.043490 29277 leader_election.cc:288] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 15:00:45.043534 29281 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [term 2 FOLLOWER]: Leader election won for term 2
I20260806 15:00:45.043656 29281 raft_consensus.cc:686] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [term 2 LEADER]: Becoming Leader. State: Replica: 409e8d7db85240958e76dda03cc6675e, State: Running, Role: LEADER
I20260806 15:00:45.043757 29281 consensus_queue.cc:237] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409e8d7db85240958e76dda03cc6675e" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44625 } }
I20260806 15:00:45.043767 29277 sys_catalog.cc:564] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:00:45.044020 29284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 409e8d7db85240958e76dda03cc6675e. Latest consensus state: current_term: 2 leader_uuid: "409e8d7db85240958e76dda03cc6675e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409e8d7db85240958e76dda03cc6675e" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44625 } } }
I20260806 15:00:45.044087 29284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [sys.catalog]: This master's current role is: LEADER
I20260806 15:00:45.044860 29282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "409e8d7db85240958e76dda03cc6675e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "409e8d7db85240958e76dda03cc6675e" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44625 } } }
I20260806 15:00:45.044891 29286 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:00:45.045447 29286 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:00:45.045590 29286 catalog_manager.cc:1259] Loaded cluster ID: ebd9ac309f93465b8ce3869eac42db29
I20260806 15:00:45.045621 29286 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:00:45.045863 29277 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 15:00:45.048051 29282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e [sys.catalog]: This master's current role is: LEADER
I20260806 15:00:45.049559 29286 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:00:45.049948 29286 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e: Loaded TSK: 0
I20260806 15:00:45.050211 29286 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 15:00:45.150456 29279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:00:45.150738 29279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:00:45.150779 29279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:00:45.152400 29279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:00:45.152460 29279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:00:45.152503 29279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 15:00:45.154590 29279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:36175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.8.84:59751/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:43639
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=34661
--tserver_master_addrs=127.11.8.126:44625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:00:45.154829 29279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:00:45.155095 29279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:00:45.162714 29335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:45.163074 29279 server_base.cc:1034] running on GCE node
W20260806 15:00:45.162881 29334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:45.163553 29337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:45.163902 29279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:00:45.164405 29279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:00:45.171332 29279 hybrid_clock.cc:648] HybridClock initialized: now 1786028445171300 us; error 46 us; skew 500 ppm
Aug 06 15:00:45 dist-test-slave-68xk krb5kdc[27825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028445, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:00:45.174741 29279 init.cc:377] Logged in from keytab as kudu/127.11.8.65@KRBTEST.COM (short username kudu)
I20260806 15:00:45.175284 29279 webserver.cc:460] Webserver started at http://127.11.8.65:34661/ using document root <none> and password file <none>
I20260806 15:00:45.175590 29279 fs_manager.cc:359] Metadata directory not provided
I20260806 15:00:45.175717 29279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 15:00:45 dist-test-slave-68xk krb5kdc[27825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028445, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for HTTP/127.11.8.84@KRBTEST.COM
I20260806 15:00:45.218820 29279 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.001s sys 0.000s
I20260806 15:00:45.220844 29346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:45.224538 29279 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260806 15:00:45.224658 29279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "898ae30bbbf34e5688ac81b6ee8bf3c1"
format_stamp: "Formatted at 2026-08-06 15:00:44 on dist-test-slave-68xk"
server_key: "71cbc366b1428dffc23278e73bf40f75"
server_key_iv: "697d439eb23f902eac1dbe4750a24e09"
server_key_version: "kuduclusterkey@0"
I20260806 15:00:45.225190 29279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:00:45.280632 29279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:00:45.283136 29279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:00:45.283308 29279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:00:45.283555 29279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:00:45.283973 29279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:00:45.284026 29279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:45.284060 29279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:00:45.284082 29279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:45.291635 29279 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:43639
I20260806 15:00:45.291716 29502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:43639 every 8 connection(s)
I20260806 15:00:45.292140 29279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
I20260806 15:00:45.296327 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 29279
I20260806 15:00:45.296644 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.8.84:59751/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:37985
--local_ip_for_outbound_sockets=127.11.8.66
--tserver_master_addrs=127.11.8.126:44625
--webserver_port=40633
--webserver_interface=127.11.8.66
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:36175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.8.84:59751/kms with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 15:00:45 dist-test-slave-68xk krb5kdc[27825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028445, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:00:45.305186 29508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:45.293147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:44625 (local address 127.11.8.65:48201)
0806 15:00:45.293364 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:45.293414 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:45.293915 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:45.294396 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:45.294401 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:45.294639 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:00:45.294827 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:45.294832 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:45.296000 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:45.296002 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:45.297384 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:45.297388 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:45.298257 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:45.298259 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:45.298404 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:45.299036 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:00:45.299047 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:00:45.300510 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:00:45.303202 (+ 2692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:45.303208 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:45.303210 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:45.303487 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:45.303817 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:45.303818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:45.303819 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:45.303955 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:45.304473 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:00:45.304479 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:00:45.304650 (+ 171us) client_negotiation.cc:769] Sending connection context
0806 15:00:45.304777 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 15:00:45.304927 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":91,"threads_started":1}
I20260806 15:00:45.305277 29509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:45.293231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:48201 (local address 127.11.8.126:44625)
0806 15:00:45.293461 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:45.293469 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:45.293504 (+ 35us) server_negotiation.cc:407] Connection header received
0806 15:00:45.294165 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:45.294171 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:45.294216 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:45.294361 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:45.294958 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:45.295887 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:45.297495 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:45.298132 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:45.298261 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:45.300644 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:45.300649 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:45.300649 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:45.300660 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:45.303099 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:45.303588 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:45.303591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:45.303592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:45.303717 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:45.304040 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:45.304041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:45.304041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:45.304381 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:45.304480 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:45.304863 (+ 383us) server_negotiation.cc:299] Negotiation successful
0806 15:00:45.305006 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":95,"threads_started":1}
I20260806 15:00:45.306589 29503 heartbeater.cc:347] Connected to a master server at 127.11.8.126:44625
I20260806 15:00:45.306701 29503 heartbeater.cc:464] Registering TS with master...
I20260806 15:00:45.306932 29503 heartbeater.cc:511] Master 127.11.8.126:44625 requested a full tablet report, sending...
W20260806 15:00:45.307962 29503 heartbeater.cc:644] Failed to heartbeat to 127.11.8.126:44625 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 15:00:45.308044 29503 heartbeater.cc:464] Registering TS with master...
I20260806 15:00:45.308125 29503 heartbeater.cc:511] Master 127.11.8.126:44625 requested a full tablet report, sending...
I20260806 15:00:45.308475 29503 heartbeater.cc:464] Registering TS with master...
I20260806 15:00:45.308557 29503 heartbeater.cc:511] Master 127.11.8.126:44625 requested a full tablet report, sending...
W20260806 15:00:45.308902 29503 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 15:00:45.400504 29510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:00:45.400695 29510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:00:45.400717 29510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:00:45.402025 29510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:00:45.402070 29510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:00:45.402097 29510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 15:00:45.403918 29510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:36175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.8.84:59751/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:37985
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=40633
--tserver_master_addrs=127.11.8.126:44625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:00:45.404155 29510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:00:45.404405 29510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:00:45.409781 29518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:45.410101 29521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:45.409786 29519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:45.411164 29510 server_base.cc:1034] running on GCE node
I20260806 15:00:45.411363 29510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:00:45.411605 29510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:00:45.412794 29510 hybrid_clock.cc:648] HybridClock initialized: now 1786028445412779 us; error 32 us; skew 500 ppm
Aug 06 15:00:45 dist-test-slave-68xk krb5kdc[27825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028445, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:00:45.416093 29510 init.cc:377] Logged in from keytab as kudu/127.11.8.66@KRBTEST.COM (short username kudu)
I20260806 15:00:45.416661 29510 webserver.cc:460] Webserver started at http://127.11.8.66:40633/ using document root <none> and password file <none>
I20260806 15:00:45.416976 29510 fs_manager.cc:359] Metadata directory not provided
I20260806 15:00:45.417068 29510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 15:00:45 dist-test-slave-68xk krb5kdc[27825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028445, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for HTTP/127.11.8.84@KRBTEST.COM
I20260806 15:00:45.436739 29510 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.000s
I20260806 15:00:45.443941 29541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:45.444135 29510 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.000s sys 0.000s
I20260806 15:00:45.444252 29510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "d06a80aa4042458f94b96a1e0d37fd5d"
format_stamp: "Formatted at 2026-08-06 15:00:44 on dist-test-slave-68xk"
server_key: "46f25be8186f42b3f7ea028ff56e9764"
server_key_iv: "6e9baaeff07f5782c781adfbb6a11406"
server_key_version: "kuduclusterkey@0"
I20260806 15:00:45.444752 29510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:00:45.467994 29510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:00:45.470384 29510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:00:45.470574 29510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:00:45.470873 29510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:00:45.471287 29510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:00:45.471340 29510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:45.471379 29510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:00:45.471398 29510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:45.480822 29710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:37985 every 8 connection(s)
I20260806 15:00:45.480939 29510 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:37985
I20260806 15:00:45.481407 29510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
I20260806 15:00:45.488754 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 29510
I20260806 15:00:45.489090 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.11.8.84:59751/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:40563
--local_ip_for_outbound_sockets=127.11.8.67
--tserver_master_addrs=127.11.8.126:44625
--webserver_port=42357
--webserver_interface=127.11.8.67
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:36175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.11.8.84:59751/kms with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 15:00:45 dist-test-slave-68xk krb5kdc[27825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028445, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:00:45.504451 29509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:45.482532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:54669 (local address 127.11.8.126:44625)
0806 15:00:45.482581 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:45.482583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:45.489698 (+ 7115us) server_negotiation.cc:407] Connection header received
0806 15:00:45.490460 (+ 762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:45.490462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:45.490476 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:45.490510 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:45.492192 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:45.492984 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:45.494556 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:45.494864 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:45.494912 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:45.500907 (+ 5995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:45.500909 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:45.500909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:45.500917 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:45.502768 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:45.503268 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:45.503269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:45.503270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:45.503328 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:45.503620 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:45.503621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:45.503622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:45.503796 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:45.503858 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:45.504255 (+ 397us) server_negotiation.cc:299] Negotiation successful
0806 15:00:45.504354 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 15:00:45.504993 29714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:45.482431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:44625 (local address 127.11.8.66:54669)
0806 15:00:45.489515 (+ 7084us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:45.489579 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:45.490294 (+ 715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:45.490716 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:45.490721 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:45.490997 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:00:45.491285 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:45.491291 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:45.493104 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:45.493105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:45.494455 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:45.494459 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:45.495023 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:45.495023 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:45.495170 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:45.495799 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:00:45.495810 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:00:45.500786 (+ 4976us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:00:45.502869 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:45.502876 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:45.502878 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:45.503174 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:45.503394 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:45.503395 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:45.503395 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:45.503540 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:45.503898 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:00:45.503906 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:00:45.504090 (+ 184us) client_negotiation.cc:769] Sending connection context
0806 15:00:45.504572 (+ 482us) client_negotiation.cc:240] Negotiation successful
0806 15:00:45.504754 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6862,"thread_start_us":85,"threads_started":1}
I20260806 15:00:45.505511 29711 heartbeater.cc:347] Connected to a master server at 127.11.8.126:44625
I20260806 15:00:45.505628 29711 heartbeater.cc:464] Registering TS with master...
I20260806 15:00:45.505839 29711 heartbeater.cc:511] Master 127.11.8.126:44625 requested a full tablet report, sending...
W20260806 15:00:45.506465 29711 heartbeater.cc:644] Failed to heartbeat to 127.11.8.126:44625 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 15:00:45.506543 29711 heartbeater.cc:464] Registering TS with master...
I20260806 15:00:45.506616 29711 heartbeater.cc:511] Master 127.11.8.126:44625 requested a full tablet report, sending...
I20260806 15:00:45.506912 29711 heartbeater.cc:464] Registering TS with master...
I20260806 15:00:45.506994 29711 heartbeater.cc:511] Master 127.11.8.126:44625 requested a full tablet report, sending...
W20260806 15:00:45.507292 29711 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 15:00:45.646909 29715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:00:45.647236 29715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:00:45.647277 29715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:00:45.649257 29715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:00:45.649317 29715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:00:45.649360 29715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 15:00:45.651779 29715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:36175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.11.8.84:59751/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:40563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=42357
--tserver_master_addrs=127.11.8.126:44625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:00:45.652041 29715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:00:45.652316 29715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:00:45.659446 29729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:45.659476 29731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:45.659726 29728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:45.660180 29715 server_base.cc:1034] running on GCE node
I20260806 15:00:45.660418 29715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:00:45.660693 29715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:00:45.662496 29715 hybrid_clock.cc:648] HybridClock initialized: now 1786028445662466 us; error 51 us; skew 500 ppm
Aug 06 15:00:45 dist-test-slave-68xk krb5kdc[27825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028445, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:00:45.666304 29715 init.cc:377] Logged in from keytab as kudu/127.11.8.67@KRBTEST.COM (short username kudu)
I20260806 15:00:45.666975 29715 webserver.cc:460] Webserver started at http://127.11.8.67:42357/ using document root <none> and password file <none>
I20260806 15:00:45.667333 29715 fs_manager.cc:359] Metadata directory not provided
I20260806 15:00:45.667447 29715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 15:00:45 dist-test-slave-68xk krb5kdc[27825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028445, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for HTTP/127.11.8.84@KRBTEST.COM
I20260806 15:00:45.687865 29715 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.001s
I20260806 15:00:45.688717 29740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:45.688930 29715 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:00:45.689020 29715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "ff01961beab84ed6b6d67c43a77ed5d9"
format_stamp: "Formatted at 2026-08-06 15:00:44 on dist-test-slave-68xk"
server_key: "6fa5d7db443d1f150ade2ee2c271ec01"
server_key_iv: "86894a322c6770d7d4282b60c74cc3df"
server_key_version: "kuduclusterkey@0"
I20260806 15:00:45.689512 29715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:00:45.710798 29715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:00:45.713528 29715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:00:45.713771 29715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:00:45.714056 29715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:00:45.714602 29715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:00:45.714704 29715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:45.714852 29715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:00:45.714922 29715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:45.722479 29715 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:40563
I20260806 15:00:45.723052 29715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
I20260806 15:00:45.725147 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 29715
I20260806 15:00:45.735272 29509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:45.725841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51450 (local address 127.11.8.126:44625)
0806 15:00:45.725895 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:45.725897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:45.726010 (+ 113us) server_negotiation.cc:407] Connection header received
0806 15:00:45.726135 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:45.726136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:45.726149 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:45.726184 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:45.727211 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:45.728121 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:45.729374 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:45.729721 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:45.729768 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:45.731776 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:45.731777 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:45.731778 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:45.731787 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:45.733696 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:45.734141 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:45.734142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:45.734143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:45.734204 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:45.734429 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:45.734430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:45.734430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:45.734603 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:45.734788 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:45.735064 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 15:00:45.735155 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 15:00:45.735263 29893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:40563 every 8 connection(s)
Aug 06 15:00:45 dist-test-slave-68xk krb5kdc[27825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028445, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:00:45.750792 29509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:45.736262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:39849 (local address 127.11.8.126:44625)
0806 15:00:45.736301 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:45.736303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:45.738730 (+ 2427us) server_negotiation.cc:407] Connection header received
0806 15:00:45.739428 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:45.739429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:45.739441 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:45.739472 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:45.740339 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:45.741067 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:45.742685 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:45.743028 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:45.743086 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:45.746299 (+ 3213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:45.746300 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:45.746301 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:45.746309 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:45.748828 (+ 2519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:45.749423 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:45.749424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:45.749425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:45.749493 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:45.749880 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:45.749881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:45.749881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:45.750050 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:45.750124 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:45.750591 (+ 467us) server_negotiation.cc:299] Negotiation successful
0806 15:00:45.750690 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:00:45.751013 29899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:45.735941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:44625 (local address 127.11.8.67:39849)
0806 15:00:45.738590 (+ 2649us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:45.738646 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:45.739304 (+ 658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:45.739680 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:45.739684 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:45.739929 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:00:45.740225 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:45.740232 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:45.741193 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:45.741194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:45.742555 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:45.742559 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:45.743265 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:45.743268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:45.743447 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:45.744148 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:00:45.744162 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:00:45.746180 (+ 2018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:00:45.748924 (+ 2744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:45.748932 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:45.748935 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:45.749319 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:45.749588 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:45.749589 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:45.749590 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:45.749785 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:45.750133 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:00:45.750141 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:00:45.750347 (+ 206us) client_negotiation.cc:769] Sending connection context
0806 15:00:45.750541 (+ 194us) client_negotiation.cc:240] Negotiation successful
0806 15:00:45.750743 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2589,"thread_start_us":116,"threads_started":1}
I20260806 15:00:45.751663 29895 heartbeater.cc:347] Connected to a master server at 127.11.8.126:44625
I20260806 15:00:45.751816 29895 heartbeater.cc:464] Registering TS with master...
I20260806 15:00:45.752068 29895 heartbeater.cc:511] Master 127.11.8.126:44625 requested a full tablet report, sending...
W20260806 15:00:45.752761 29895 heartbeater.cc:644] Failed to heartbeat to 127.11.8.126:44625 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 15:00:45.752892 29895 heartbeater.cc:464] Registering TS with master...
I20260806 15:00:45.752995 29895 heartbeater.cc:511] Master 127.11.8.126:44625 requested a full tablet report, sending...
I20260806 15:00:45.753329 29895 heartbeater.cc:464] Registering TS with master...
I20260806 15:00:45.753443 29895 heartbeater.cc:511] Master 127.11.8.126:44625 requested a full tablet report, sending...
W20260806 15:00:45.753772 29895 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 15:00:46 dist-test-slave-68xk krb5kdc[27825](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786028446, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:00:46.309075 29503 heartbeater.cc:464] Registering TS with master...
I20260806 15:00:46.309240 29503 heartbeater.cc:511] Master 127.11.8.126:44625 requested a full tablet report, sending...
I20260806 15:00:46.309628 29230 ts_manager.cc:194] Registered new tserver with Master: 898ae30bbbf34e5688ac81b6ee8bf3c1 (127.11.8.65:43639)
I20260806 15:00:46.310726 29230 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.65@KRBTEST.COM'} at 127.11.8.65:48201
I20260806 15:00:46.507476 29711 heartbeater.cc:464] Registering TS with master...
I20260806 15:00:46.507642 29711 heartbeater.cc:511] Master 127.11.8.126:44625 requested a full tablet report, sending...
I20260806 15:00:46.507972 29230 ts_manager.cc:194] Registered new tserver with Master: d06a80aa4042458f94b96a1e0d37fd5d (127.11.8.66:37985)
I20260806 15:00:46.508543 29230 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.66@KRBTEST.COM'} at 127.11.8.66:54669
Aug 06 15:00:46 dist-test-slave-68xk krb5kdc[27825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028446, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for HTTP/127.11.8.84@KRBTEST.COM
I20260806 15:00:46.757488 29895 heartbeater.cc:464] Registering TS with master...
I20260806 15:00:46.757660 29895 heartbeater.cc:511] Master 127.11.8.126:44625 requested a full tablet report, sending...
I20260806 15:00:46.757968 29230 ts_manager.cc:194] Registered new tserver with Master: ff01961beab84ed6b6d67c43a77ed5d9 (127.11.8.67:40563)
I20260806 15:00:46.758518 29230 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.67@KRBTEST.COM'} at 127.11.8.67:39849
I20260806 15:00:46.762068 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:00:46.775666 29923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:46.763661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51466 (local address 127.11.8.126:44625)
0806 15:00:46.764454 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:46.764456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:46.764463 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:00:46.764485 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:46.764486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:46.764499 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:46.764597 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:46.765411 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:46.766178 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:46.767237 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:46.767544 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:46.767608 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:46.769481 (+ 1873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:46.769483 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:46.769483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:46.769491 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:46.772844 (+ 3353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:46.773506 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:46.773507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:46.773508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:46.773623 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:46.773976 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:46.773978 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:46.773978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:46.774212 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:46.774347 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:46.775323 (+ 976us) server_negotiation.cc:299] Negotiation successful
0806 15:00:46.775465 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":772,"thread_start_us":105,"threads_started":1}
I20260806 15:00:46.778466 29230 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51466:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:00:46.780894 29230 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:00:46.794863 29936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:46.786868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:40563 (local address 127.0.0.1:47036)
0806 15:00:46.789003 (+ 2135us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:46.789015 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:46.789110 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:46.789581 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:46.789587 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:46.789590 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:00:46.789794 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:46.789797 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:46.791180 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:46.791181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:46.792804 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:46.792807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:46.794492 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:46.794493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:46.794617 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:46.794636 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 15:00:46.794706 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 15:00:46.794772 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2088,"mutex_wait_us":33,"thread_start_us":293,"threads_started":1}
I20260806 15:00:46.795574 29935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:46.787008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47036 (local address 127.11.8.67:40563)
0806 15:00:46.788311 (+ 1303us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:46.788316 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:46.789189 (+ 873us) server_negotiation.cc:407] Connection header received
0806 15:00:46.789235 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:46.789238 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:46.789258 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:46.789349 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:00:46.789891 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:46.791083 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:46.792929 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:46.794067 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:46.794221 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:46.794331 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:46.795389 (+ 1058us) server_negotiation.cc:299] Negotiation successful
0806 15:00:46.795445 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1264,"thread_start_us":91,"threads_started":1}
I20260806 15:00:46.797125 29806 tablet_service.cc:1467] Processing CreateTablet for tablet 4055f8d62dc74446a1ad0137b6d8a8a8 (DEFAULT_TABLE table=test-table [id=5816a58b0f974e34a91b155a052f2bb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:00:46.797585 29806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4055f8d62dc74446a1ad0137b6d8a8a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:00:46.799150 29941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:46.786797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:37985 (local address 127.0.0.1:43448)
0806 15:00:46.788462 (+ 1665us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:46.788510 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:46.788771 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:46.789888 (+ 1117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:46.789889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:46.789891 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:00:46.789941 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:46.789942 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:46.791438 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:46.791439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:46.792786 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:46.792790 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:46.798858 (+ 6068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:46.798859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:46.798931 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:46.798935 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:00:46.799000 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 15:00:46.799069 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1601,"thread_start_us":132,"threads_started":1}
I20260806 15:00:46.799625 29943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:46.787548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43448 (local address 127.11.8.66:37985)
0806 15:00:46.787952 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:46.787956 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:46.788741 (+ 785us) server_negotiation.cc:407] Connection header received
0806 15:00:46.788913 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:46.788916 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:46.788933 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:46.789256 (+ 323us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:00:46.790047 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:46.791322 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:46.794258 (+ 2936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:46.798604 (+ 4346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:46.798703 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:46.798820 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:46.799483 (+ 663us) server_negotiation.cc:299] Negotiation successful
0806 15:00:46.799525 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":331,"threads_started":1}
I20260806 15:00:46.800983 29612 tablet_service.cc:1467] Processing CreateTablet for tablet 4055f8d62dc74446a1ad0137b6d8a8a8 (DEFAULT_TABLE table=test-table [id=5816a58b0f974e34a91b155a052f2bb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:00:46.801613 29612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4055f8d62dc74446a1ad0137b6d8a8a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:00:46.802126 29939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:46.787300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50624 (local address 127.11.8.65:43639)
0806 15:00:46.787479 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:46.787942 (+ 463us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:46.793214 (+ 5272us) server_negotiation.cc:407] Connection header received
0806 15:00:46.793372 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:46.793375 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:46.793396 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:46.793693 (+ 297us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:00:46.795222 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:46.796951 (+ 1729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:46.800363 (+ 3412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:46.801384 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:46.801481 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:46.801539 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:46.801952 (+ 413us) server_negotiation.cc:299] Negotiation successful
0806 15:00:46.802002 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":65,"threads_started":1}
I20260806 15:00:46.802399 29934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:46.787079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:43639 (local address 127.0.0.1:50624)
0806 15:00:46.792926 (+ 5847us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:46.792937 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:46.793256 (+ 319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:46.793934 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:46.793935 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:46.793937 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:00:46.794159 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:46.794162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:46.797082 (+ 2920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:46.797083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:46.800254 (+ 3171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:46.800257 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:46.801622 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:46.801623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:46.801780 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:46.801786 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 15:00:46.802246 (+ 460us) client_negotiation.cc:240] Negotiation successful
0806 15:00:46.802323 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5828,"thread_start_us":324,"threads_started":1}
I20260806 15:00:46.803421 29411 tablet_service.cc:1467] Processing CreateTablet for tablet 4055f8d62dc74446a1ad0137b6d8a8a8 (DEFAULT_TABLE table=test-table [id=5816a58b0f974e34a91b155a052f2bb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:00:46.803907 29411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4055f8d62dc74446a1ad0137b6d8a8a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:00:46.805795 29946 tablet_bootstrap.cc:492] T 4055f8d62dc74446a1ad0137b6d8a8a8 P ff01961beab84ed6b6d67c43a77ed5d9: Bootstrap starting.
I20260806 15:00:46.809584 29946 tablet_bootstrap.cc:654] T 4055f8d62dc74446a1ad0137b6d8a8a8 P ff01961beab84ed6b6d67c43a77ed5d9: Neither blocks nor log segments found. Creating new log.
I20260806 15:00:46.810993 29946 log.cc:824] T 4055f8d62dc74446a1ad0137b6d8a8a8 P ff01961beab84ed6b6d67c43a77ed5d9: Log is configured to *not* fsync() on all Append() calls
I20260806 15:00:46.810993 29948 tablet_bootstrap.cc:492] T 4055f8d62dc74446a1ad0137b6d8a8a8 P 898ae30bbbf34e5688ac81b6ee8bf3c1: Bootstrap starting.
I20260806 15:00:46.811739 29947 tablet_bootstrap.cc:492] T 4055f8d62dc74446a1ad0137b6d8a8a8 P d06a80aa4042458f94b96a1e0d37fd5d: Bootstrap starting.
I20260806 15:00:46.812708 29948 tablet_bootstrap.cc:654] T 4055f8d62dc74446a1ad0137b6d8a8a8 P 898ae30bbbf34e5688ac81b6ee8bf3c1: Neither blocks nor log segments found. Creating new log.
I20260806 15:00:46.813524 29947 tablet_bootstrap.cc:654] T 4055f8d62dc74446a1ad0137b6d8a8a8 P d06a80aa4042458f94b96a1e0d37fd5d: Neither blocks nor log segments found. Creating new log.
I20260806 15:00:46.813724 29948 log.cc:824] T 4055f8d62dc74446a1ad0137b6d8a8a8 P 898ae30bbbf34e5688ac81b6ee8bf3c1: Log is configured to *not* fsync() on all Append() calls
I20260806 15:00:46.814530 29947 log.cc:824] T 4055f8d62dc74446a1ad0137b6d8a8a8 P d06a80aa4042458f94b96a1e0d37fd5d: Log is configured to *not* fsync() on all Append() calls
I20260806 15:00:46.815346 29946 tablet_bootstrap.cc:492] T 4055f8d62dc74446a1ad0137b6d8a8a8 P ff01961beab84ed6b6d67c43a77ed5d9: No bootstrap required, opened a new log
I20260806 15:00:46.815435 29946 ts_tablet_manager.cc:1397] T 4055f8d62dc74446a1ad0137b6d8a8a8 P ff01961beab84ed6b6d67c43a77ed5d9: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.004s
I20260806 15:00:46.815727 29947 tablet_bootstrap.cc:492] T 4055f8d62dc74446a1ad0137b6d8a8a8 P d06a80aa4042458f94b96a1e0d37fd5d: No bootstrap required, opened a new log
I20260806 15:00:46.815806 29947 ts_tablet_manager.cc:1397] T 4055f8d62dc74446a1ad0137b6d8a8a8 P d06a80aa4042458f94b96a1e0d37fd5d: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260806 15:00:46.821010 29946 raft_consensus.cc:348] T 4055f8d62dc74446a1ad0137b6d8a8a8 P ff01961beab84ed6b6d67c43a77ed5d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06a80aa4042458f94b96a1e0d37fd5d" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 37985 } } peers { permanent_uuid: "898ae30bbbf34e5688ac81b6ee8bf3c1" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43639 } } peers { permanent_uuid: "ff01961beab84ed6b6d67c43a77ed5d9" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40563 } }
I20260806 15:00:46.821259 29946 raft_consensus.cc:374] T 4055f8d62dc74446a1ad0137b6d8a8a8 P ff01961beab84ed6b6d67c43a77ed5d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:00:46.821331 29946 raft_consensus.cc:729] T 4055f8d62dc74446a1ad0137b6d8a8a8 P ff01961beab84ed6b6d67c43a77ed5d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff01961beab84ed6b6d67c43a77ed5d9, State: Initialized, Role: FOLLOWER
I20260806 15:00:46.821555 29946 consensus_queue.cc:260] T 4055f8d62dc74446a1ad0137b6d8a8a8 P ff01961beab84ed6b6d67c43a77ed5d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06a80aa4042458f94b96a1e0d37fd5d" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 37985 } } peers { permanent_uuid: "898ae30bbbf34e5688ac81b6ee8bf3c1" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43639 } } peers { permanent_uuid: "ff01961beab84ed6b6d67c43a77ed5d9" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40563 } }
I20260806 15:00:46.822094 29946 ts_tablet_manager.cc:1428] T 4055f8d62dc74446a1ad0137b6d8a8a8 P ff01961beab84ed6b6d67c43a77ed5d9: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260806 15:00:46.822247 29895 heartbeater.cc:503] Master 127.11.8.126:44625 was elected leader, sending a full tablet report...
I20260806 15:00:46.822441 29947 raft_consensus.cc:348] T 4055f8d62dc74446a1ad0137b6d8a8a8 P d06a80aa4042458f94b96a1e0d37fd5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06a80aa4042458f94b96a1e0d37fd5d" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 37985 } } peers { permanent_uuid: "898ae30bbbf34e5688ac81b6ee8bf3c1" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43639 } } peers { permanent_uuid: "ff01961beab84ed6b6d67c43a77ed5d9" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40563 } }
I20260806 15:00:46.822641 29947 raft_consensus.cc:374] T 4055f8d62dc74446a1ad0137b6d8a8a8 P d06a80aa4042458f94b96a1e0d37fd5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:00:46.822696 29947 raft_consensus.cc:729] T 4055f8d62dc74446a1ad0137b6d8a8a8 P d06a80aa4042458f94b96a1e0d37fd5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d06a80aa4042458f94b96a1e0d37fd5d, State: Initialized, Role: FOLLOWER
I20260806 15:00:46.822863 29947 consensus_queue.cc:260] T 4055f8d62dc74446a1ad0137b6d8a8a8 P d06a80aa4042458f94b96a1e0d37fd5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06a80aa4042458f94b96a1e0d37fd5d" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 37985 } } peers { permanent_uuid: "898ae30bbbf34e5688ac81b6ee8bf3c1" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43639 } } peers { permanent_uuid: "ff01961beab84ed6b6d67c43a77ed5d9" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40563 } }
I20260806 15:00:46.823351 29947 ts_tablet_manager.cc:1428] T 4055f8d62dc74446a1ad0137b6d8a8a8 P d06a80aa4042458f94b96a1e0d37fd5d: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260806 15:00:46.823505 29711 heartbeater.cc:503] Master 127.11.8.126:44625 was elected leader, sending a full tablet report...
I20260806 15:00:46.825240 29948 tablet_bootstrap.cc:492] T 4055f8d62dc74446a1ad0137b6d8a8a8 P 898ae30bbbf34e5688ac81b6ee8bf3c1: No bootstrap required, opened a new log
I20260806 15:00:46.825338 29948 ts_tablet_manager.cc:1397] T 4055f8d62dc74446a1ad0137b6d8a8a8 P 898ae30bbbf34e5688ac81b6ee8bf3c1: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.000s
I20260806 15:00:46.830392 29948 raft_consensus.cc:348] T 4055f8d62dc74446a1ad0137b6d8a8a8 P 898ae30bbbf34e5688ac81b6ee8bf3c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06a80aa4042458f94b96a1e0d37fd5d" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 37985 } } peers { permanent_uuid: "898ae30bbbf34e5688ac81b6ee8bf3c1" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43639 } } peers { permanent_uuid: "ff01961beab84ed6b6d67c43a77ed5d9" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40563 } }
I20260806 15:00:46.830641 29948 raft_consensus.cc:374] T 4055f8d62dc74446a1ad0137b6d8a8a8 P 898ae30bbbf34e5688ac81b6ee8bf3c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:00:46.830689 29948 raft_consensus.cc:729] T 4055f8d62dc74446a1ad0137b6d8a8a8 P 898ae30bbbf34e5688ac81b6ee8bf3c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 898ae30bbbf34e5688ac81b6ee8bf3c1, State: Initialized, Role: FOLLOWER
I20260806 15:00:46.830832 29948 consensus_queue.cc:260] T 4055f8d62dc74446a1ad0137b6d8a8a8 P 898ae30bbbf34e5688ac81b6ee8bf3c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06a80aa4042458f94b96a1e0d37fd5d" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 37985 } } peers { permanent_uuid: "898ae30bbbf34e5688ac81b6ee8bf3c1" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43639 } } peers { permanent_uuid: "ff01961beab84ed6b6d67c43a77ed5d9" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40563 } }
I20260806 15:00:46.831241 29948 ts_tablet_manager.cc:1428] T 4055f8d62dc74446a1ad0137b6d8a8a8 P 898ae30bbbf34e5688ac81b6ee8bf3c1: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260806 15:00:46.831372 29503 heartbeater.cc:503] Master 127.11.8.126:44625 was elected leader, sending a full tablet report...
Aug 06 15:00:46 dist-test-slave-68xk krb5kdc[27825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028446, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for HTTP/127.11.8.84@KRBTEST.COM
I20260806 15:00:46.883214 29957 raft_consensus.cc:482] T 4055f8d62dc74446a1ad0137b6d8a8a8 P 898ae30bbbf34e5688ac81b6ee8bf3c1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:00:46.883478 29957 raft_consensus.cc:504] T 4055f8d62dc74446a1ad0137b6d8a8a8 P 898ae30bbbf34e5688ac81b6ee8bf3c1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06a80aa4042458f94b96a1e0d37fd5d" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 37985 } } peers { permanent_uuid: "898ae30bbbf34e5688ac81b6ee8bf3c1" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43639 } } peers { permanent_uuid: "ff01961beab84ed6b6d67c43a77ed5d9" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40563 } }
I20260806 15:00:46.884048 29957 leader_election.cc:288] T 4055f8d62dc74446a1ad0137b6d8a8a8 P 898ae30bbbf34e5688ac81b6ee8bf3c1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d06a80aa4042458f94b96a1e0d37fd5d (127.11.8.66:37985), ff01961beab84ed6b6d67c43a77ed5d9 (127.11.8.67:40563)
I20260806 15:00:46.889074 29943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:46.884748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:33773 (local address 127.11.8.66:37985)
0806 15:00:46.884831 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:46.884833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:46.884932 (+ 99us) server_negotiation.cc:407] Connection header received
0806 15:00:46.885092 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:46.885093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:46.885108 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:46.885151 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:00:46.885621 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:46.886347 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:46.887979 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:46.888659 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:46.888713 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:46.888725 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:46.888958 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 15:00:46.889002 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 15:00:46.889312 29959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:46.884340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:37985 (local address 127.11.8.65:33773)
0806 15:00:46.884654 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:46.884669 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:46.884965 (+ 296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:46.885271 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:46.885272 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:46.885274 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:00:46.885536 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:46.885539 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:46.886458 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:46.886459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:46.887872 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:46.887875 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:46.888794 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:46.888795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:46.888868 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:46.888873 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 15:00:46.889162 (+ 289us) client_negotiation.cc:240] Negotiation successful
0806 15:00:46.889207 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":236,"threads_started":1}
I20260806 15:00:46.890790 29935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:46.885383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:34177 (local address 127.11.8.67:40563)
0806 15:00:46.885453 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:46.885455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:46.886854 (+ 1399us) server_negotiation.cc:407] Connection header received
0806 15:00:46.886967 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:46.886969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:46.886982 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:46.887011 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:00:46.887227 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:46.887885 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:46.889472 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:46.890098 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:46.890173 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:46.890186 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:46.890673 (+ 487us) server_negotiation.cc:299] Negotiation successful
0806 15:00:46.890722 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 15:00:46.890873 29623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4055f8d62dc74446a1ad0137b6d8a8a8" candidate_uuid: "898ae30bbbf34e5688ac81b6ee8bf3c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d06a80aa4042458f94b96a1e0d37fd5d" is_pre_election: true
I20260806 15:00:46.891083 29962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:46.884987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:40563 (local address 127.11.8.65:34177)
0806 15:00:46.886554 (+ 1567us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:46.886567 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:46.886891 (+ 324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:46.887084 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:46.887085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:46.887087 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:00:46.887161 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:46.887162 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:46.887983 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:46.887984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:46.889360 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:46.889364 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:46.890343 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:46.890345 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:46.890569 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:46.890583 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 15:00:46.890882 (+ 299us) client_negotiation.cc:240] Negotiation successful
0806 15:00:46.890934 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1545,"thread_start_us":278,"threads_started":1}
I20260806 15:00:46.891091 29623 raft_consensus.cc:2454] T 4055f8d62dc74446a1ad0137b6d8a8a8 P d06a80aa4042458f94b96a1e0d37fd5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 898ae30bbbf34e5688ac81b6ee8bf3c1 in term 0.
I20260806 15:00:46.891674 29349 leader_election.cc:302] T 4055f8d62dc74446a1ad0137b6d8a8a8 P 898ae30bbbf34e5688ac81b6ee8bf3c1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 898ae30bbbf34e5688ac81b6ee8bf3c1, d06a80aa4042458f94b96a1e0d37fd5d; no voters:
I20260806 15:00:46.891803 29828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4055f8d62dc74446a1ad0137b6d8a8a8" candidate_uuid: "898ae30bbbf34e5688ac81b6ee8bf3c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff01961beab84ed6b6d67c43a77ed5d9" is_pre_election: true
I20260806 15:00:46.891937 29828 raft_consensus.cc:2454] T 4055f8d62dc74446a1ad0137b6d8a8a8 P ff01961beab84ed6b6d67c43a77ed5d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 898ae30bbbf34e5688ac81b6ee8bf3c1 in term 0.
I20260806 15:00:46.892052 29957 raft_consensus.cc:2781] T 4055f8d62dc74446a1ad0137b6d8a8a8 P 898ae30bbbf34e5688ac81b6ee8bf3c1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:00:46.892113 29957 raft_consensus.cc:482] T 4055f8d62dc74446a1ad0137b6d8a8a8 P 898ae30bbbf34e5688ac81b6ee8bf3c1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:00:46.892139 29957 raft_consensus.cc:3037] T 4055f8d62dc74446a1ad0137b6d8a8a8 P 898ae30bbbf34e5688ac81b6ee8bf3c1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:00:46.893257 29957 raft_consensus.cc:504] T 4055f8d62dc74446a1ad0137b6d8a8a8 P 898ae30bbbf34e5688ac81b6ee8bf3c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06a80aa4042458f94b96a1e0d37fd5d" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 37985 } } peers { permanent_uuid: "898ae30bbbf34e5688ac81b6ee8bf3c1" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43639 } } peers { permanent_uuid: "ff01961beab84ed6b6d67c43a77ed5d9" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40563 } }
I20260806 15:00:46.893410 29957 leader_election.cc:288] T 4055f8d62dc74446a1ad0137b6d8a8a8 P 898ae30bbbf34e5688ac81b6ee8bf3c1 [CANDIDATE]: Term 1 election: Requested vote from peers d06a80aa4042458f94b96a1e0d37fd5d (127.11.8.66:37985), ff01961beab84ed6b6d67c43a77ed5d9 (127.11.8.67:40563)
I20260806 15:00:46.893615 29828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4055f8d62dc74446a1ad0137b6d8a8a8" candidate_uuid: "898ae30bbbf34e5688ac81b6ee8bf3c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff01961beab84ed6b6d67c43a77ed5d9"
I20260806 15:00:46.893709 29828 raft_consensus.cc:3037] T 4055f8d62dc74446a1ad0137b6d8a8a8 P ff01961beab84ed6b6d67c43a77ed5d9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:00:46.894802 29828 raft_consensus.cc:2454] T 4055f8d62dc74446a1ad0137b6d8a8a8 P ff01961beab84ed6b6d67c43a77ed5d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 898ae30bbbf34e5688ac81b6ee8bf3c1 in term 1.
I20260806 15:00:46.895040 29350 leader_election.cc:302] T 4055f8d62dc74446a1ad0137b6d8a8a8 P 898ae30bbbf34e5688ac81b6ee8bf3c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 898ae30bbbf34e5688ac81b6ee8bf3c1, ff01961beab84ed6b6d67c43a77ed5d9; no voters:
I20260806 15:00:46.895092 29623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4055f8d62dc74446a1ad0137b6d8a8a8" candidate_uuid: "898ae30bbbf34e5688ac81b6ee8bf3c1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d06a80aa4042458f94b96a1e0d37fd5d"
I20260806 15:00:46.895174 29623 raft_consensus.cc:3037] T 4055f8d62dc74446a1ad0137b6d8a8a8 P d06a80aa4042458f94b96a1e0d37fd5d [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:00:46.896318 29623 raft_consensus.cc:2454] T 4055f8d62dc74446a1ad0137b6d8a8a8 P d06a80aa4042458f94b96a1e0d37fd5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 898ae30bbbf34e5688ac81b6ee8bf3c1 in term 1.
I20260806 15:00:46.896507 29957 raft_consensus.cc:2781] T 4055f8d62dc74446a1ad0137b6d8a8a8 P 898ae30bbbf34e5688ac81b6ee8bf3c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:00:46.896847 29957 raft_consensus.cc:686] T 4055f8d62dc74446a1ad0137b6d8a8a8 P 898ae30bbbf34e5688ac81b6ee8bf3c1 [term 1 LEADER]: Becoming Leader. State: Replica: 898ae30bbbf34e5688ac81b6ee8bf3c1, State: Running, Role: LEADER
I20260806 15:00:46.897019 29957 consensus_queue.cc:237] T 4055f8d62dc74446a1ad0137b6d8a8a8 P 898ae30bbbf34e5688ac81b6ee8bf3c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06a80aa4042458f94b96a1e0d37fd5d" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 37985 } } peers { permanent_uuid: "898ae30bbbf34e5688ac81b6ee8bf3c1" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43639 } } peers { permanent_uuid: "ff01961beab84ed6b6d67c43a77ed5d9" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40563 } }
I20260806 15:00:46.898263 29228 catalog_manager.cc:5495] T 4055f8d62dc74446a1ad0137b6d8a8a8 P 898ae30bbbf34e5688ac81b6ee8bf3c1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 898ae30bbbf34e5688ac81b6ee8bf3c1 (127.11.8.65). New cstate: current_term: 1 leader_uuid: "898ae30bbbf34e5688ac81b6ee8bf3c1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d06a80aa4042458f94b96a1e0d37fd5d" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 37985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "898ae30bbbf34e5688ac81b6ee8bf3c1" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 43639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff01961beab84ed6b6d67c43a77ed5d9" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40563 } health_report { overall_health: UNKNOWN } } }
I20260806 15:00:46.913264 29939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:46.909112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50628 (local address 127.11.8.65:43639)
0806 15:00:46.909167 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:46.909170 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:46.909215 (+ 45us) server_negotiation.cc:407] Connection header received
0806 15:00:46.909330 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:46.909331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:46.909353 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:46.909448 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:00:46.909617 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:46.910449 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:46.911617 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:46.911916 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:46.911980 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:46.912610 (+ 630us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:00:46.912748 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:00:46.912826 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:46.913157 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 15:00:46.913183 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 15:00:46.915598 29623 raft_consensus.cc:1261] T 4055f8d62dc74446a1ad0137b6d8a8a8 P d06a80aa4042458f94b96a1e0d37fd5d [term 1 FOLLOWER]: Refusing update from remote peer 898ae30bbbf34e5688ac81b6ee8bf3c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:00:46.915643 29828 raft_consensus.cc:1261] T 4055f8d62dc74446a1ad0137b6d8a8a8 P ff01961beab84ed6b6d67c43a77ed5d9 [term 1 FOLLOWER]: Refusing update from remote peer 898ae30bbbf34e5688ac81b6ee8bf3c1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:00:46.916006 29964 consensus_queue.cc:1035] T 4055f8d62dc74446a1ad0137b6d8a8a8 P 898ae30bbbf34e5688ac81b6ee8bf3c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d06a80aa4042458f94b96a1e0d37fd5d" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 37985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:00:46.916173 29964 consensus_queue.cc:1035] T 4055f8d62dc74446a1ad0137b6d8a8a8 P 898ae30bbbf34e5688ac81b6ee8bf3c1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff01961beab84ed6b6d67c43a77ed5d9" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:00:46.920171 29972 mvcc.cc:205] Tried to move back new op lower bound from 7315572518562709504 to 7315572518491676672. Current Snapshot: MvccSnapshot[applied={T|T < 7315572518562709504}]
I20260806 15:00:46.920933 29969 mvcc.cc:205] Tried to move back new op lower bound from 7315572518562709504 to 7315572518491676672. Current Snapshot: MvccSnapshot[applied={T|T < 7315572518562709504 or (T in {7315572518562709504})}]
I20260806 15:00:46.921973 29228 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:00:46.922088 29228 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:00:46.922926 29970 mvcc.cc:205] Tried to move back new op lower bound from 7315572518562709504 to 7315572518491676672. Current Snapshot: MvccSnapshot[applied={T|T < 7315572518562709504}]
I20260806 15:00:46.923272 29228 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 409e8d7db85240958e76dda03cc6675e: Sending DeleteTablet for 3 replicas of tablet 4055f8d62dc74446a1ad0137b6d8a8a8
I20260806 15:00:46.924440 29806 tablet_service.cc:1514] Processing DeleteTablet for tablet 4055f8d62dc74446a1ad0137b6d8a8a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:00:46 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:47036
I20260806 15:00:46.925259 29612 tablet_service.cc:1514] Processing DeleteTablet for tablet 4055f8d62dc74446a1ad0137b6d8a8a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:00:46 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:43448
I20260806 15:00:46.925886 29411 tablet_service.cc:1514] Processing DeleteTablet for tablet 4055f8d62dc74446a1ad0137b6d8a8a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:00:46 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:50624
I20260806 15:00:46.926971 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 29279
I20260806 15:00:46.937666 29984 tablet_replica.cc:330] T 4055f8d62dc74446a1ad0137b6d8a8a8 P d06a80aa4042458f94b96a1e0d37fd5d: stopping tablet replica
I20260806 15:00:46.937851 29984 raft_consensus.cc:2229] T 4055f8d62dc74446a1ad0137b6d8a8a8 P d06a80aa4042458f94b96a1e0d37fd5d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:00:46.937968 29984 raft_consensus.cc:2258] T 4055f8d62dc74446a1ad0137b6d8a8a8 P d06a80aa4042458f94b96a1e0d37fd5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:00:46.942725 29983 tablet_replica.cc:330] T 4055f8d62dc74446a1ad0137b6d8a8a8 P ff01961beab84ed6b6d67c43a77ed5d9: stopping tablet replica
I20260806 15:00:46.942871 29983 raft_consensus.cc:2229] T 4055f8d62dc74446a1ad0137b6d8a8a8 P ff01961beab84ed6b6d67c43a77ed5d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:00:46.943010 29983 raft_consensus.cc:2258] T 4055f8d62dc74446a1ad0137b6d8a8a8 P ff01961beab84ed6b6d67c43a77ed5d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:00:46.943604 29983 ts_tablet_manager.cc:1905] T 4055f8d62dc74446a1ad0137b6d8a8a8 P ff01961beab84ed6b6d67c43a77ed5d9: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 15:00:46.945734 29211 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.65:43639 (error 108)
I20260806 15:00:46.946151 29983 ts_tablet_manager.cc:1918] T 4055f8d62dc74446a1ad0137b6d8a8a8 P ff01961beab84ed6b6d67c43a77ed5d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:00:46.946305 29984 ts_tablet_manager.cc:1905] T 4055f8d62dc74446a1ad0137b6d8a8a8 P d06a80aa4042458f94b96a1e0d37fd5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:00:46.946380 29983 log.cc:1196] T 4055f8d62dc74446a1ad0137b6d8a8a8 P ff01961beab84ed6b6d67c43a77ed5d9: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-2/wal/wals/4055f8d62dc74446a1ad0137b6d8a8a8
I20260806 15:00:46.946736 29983 ts_tablet_manager.cc:1939] T 4055f8d62dc74446a1ad0137b6d8a8a8 P ff01961beab84ed6b6d67c43a77ed5d9: Deleting consensus metadata
I20260806 15:00:46.947268 29213 catalog_manager.cc:4841] TS ff01961beab84ed6b6d67c43a77ed5d9 (127.11.8.67:40563): tablet 4055f8d62dc74446a1ad0137b6d8a8a8 (table test-table [id=5816a58b0f974e34a91b155a052f2bb6]) successfully deleted
I20260806 15:00:46.948028 29934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:46.947898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:43639 (local address 127.0.0.1:50642)
0806 15:00:46.947954 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:46.947990 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.65:43639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 15:00:46.948179 29211 catalog_manager.cc:4584] TS 898ae30bbbf34e5688ac81b6ee8bf3c1 (127.11.8.65:43639): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4055f8d62dc74446a1ad0137b6d8a8a8: Network error: Client connection negotiation failed: client connection to 127.11.8.65:43639: connect: Connection refused (error 111)
I20260806 15:00:46.948390 29984 ts_tablet_manager.cc:1918] T 4055f8d62dc74446a1ad0137b6d8a8a8 P d06a80aa4042458f94b96a1e0d37fd5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:00:46.948434 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 29510
I20260806 15:00:46.948470 29984 log.cc:1196] T 4055f8d62dc74446a1ad0137b6d8a8a8 P d06a80aa4042458f94b96a1e0d37fd5d: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786028287744255-11297-0/minicluster-data/ts-1/wal/wals/4055f8d62dc74446a1ad0137b6d8a8a8
W20260806 15:00:46.978892 29934 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:00:46.978676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:37985 (local address 127.0.0.1:43462)
0806 15:00:46.978748 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:46.978761 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:46.978817 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.66:37985: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 15:00:46.979229 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 29715
I20260806 15:00:47.001102 29934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:47.000962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:43639 (local address 127.0.0.1:50648)
0806 15:00:47.001018 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:47.001052 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.65:43639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 15:00:47.001827 29934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:47.001736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:43639 (local address 127.0.0.1:50656)
0806 15:00:47.001774 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:47.001798 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.65:43639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260806 15:00:47.022594 29934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:47.022457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:37985 (local address 127.0.0.1:43470)
0806 15:00:47.022513 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:47.022545 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.66:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 15:00:47.023005 29934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:47.022927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:37985 (local address 127.0.0.1:43486)
0806 15:00:47.022961 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:47.022981 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.66:37985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260806 15:00:47.049657 29934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:47.049503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:43639 (local address 127.0.0.1:50672)
0806 15:00:47.049563 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:47.049596 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.65:43639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 15:00:47.050077 29934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:47.049992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:43639 (local address 127.0.0.1:50674)
0806 15:00:47.050031 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:47.050053 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.65:43639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 15:00:47.054225 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 29187
I20260806 15:00:47.063848 11297 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 15:00:47.408003 11297 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 15:00:47.408146 11297 mini_ranger.cc:66] Stopping Ranger...
I20260806 15:00:47.752559 11297 mini_ranger.cc:68] Stopped Ranger
2026-08-06 15:00:47.752 UTC [27859] LOG: received smart shutdown request
2026-08-06 15:00:47.757 UTC [27859] LOG: background worker "logical replication launcher" (PID 27870) exited with exit code 1
2026-08-06 15:00:47.757 UTC [27865] LOG: shutting down
2026-08-06 15:00:47.935 UTC [27859] LOG: database system is shut down
2026-08-06T15:00:47Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50171 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:00:48 dist-test-slave-68xk krb5kdc[30001](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:00:48 dist-test-slave-68xk krb5kdc[30001](info): set up 2 sockets
Aug 06 15:00:48 dist-test-slave-68xk krb5kdc[30001](info): commencing operation
krb5kdc: starting...
W20260806 15:00:49.267517 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 15:00:49 dist-test-slave-68xk krb5kdc[30001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:00:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:00:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 15:00:49.390321 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:35355
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:39303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:35355,127.11.8.125:32827,127.11.8.124:36559
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:00:49.472211 30021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:00:49.472453 30021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:00:49.472491 30021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:00:49.473636 30021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:00:49.473704 30021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:00:49.473735 30021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:00:49.473759 30021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:00:49.473796 30021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:00:49.475340 30021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:39303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:35355,127.11.8.125:32827,127.11.8.124:36559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:35355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:00:49.475607 30021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:00:49.475842 30021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:00:49.480935 30028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:49.480935 30027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:49.480964 30021 server_base.cc:1034] running on GCE node
W20260806 15:00:49.480939 30030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:49.481518 30021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:00:49.481827 30021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:00:49.482975 30021 hybrid_clock.cc:648] HybridClock initialized: now 1786028449482934 us; error 67 us; skew 500 ppm
Aug 06 15:00:49 dist-test-slave-68xk krb5kdc[30001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028449, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:00:49.491063 30021 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 15:00:49.491526 30021 webserver.cc:460] Webserver started at http://127.11.8.126:42717/ using document root <none> and password file <none>
I20260806 15:00:49.491779 30021 fs_manager.cc:359] Metadata directory not provided
I20260806 15:00:49.491847 30021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:00:49.491990 30021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:00:49.493057 30021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "9fa69fcac81740f3bd1e22254fc5fbd1"
format_stamp: "Formatted at 2026-08-06 15:00:49 on dist-test-slave-68xk"
server_key: "76d3a5b30ab389bd3f32ccd61600fc97"
server_key_iv: "a08964f2fcd8223bb75c8c055c2c562f"
server_key_version: "encryptionkey@0"
I20260806 15:00:49.493422 30021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "9fa69fcac81740f3bd1e22254fc5fbd1"
format_stamp: "Formatted at 2026-08-06 15:00:49 on dist-test-slave-68xk"
server_key: "76d3a5b30ab389bd3f32ccd61600fc97"
server_key_iv: "a08964f2fcd8223bb75c8c055c2c562f"
server_key_version: "encryptionkey@0"
I20260806 15:00:49.494755 30021 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:00:49.495491 30047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:49.495659 30021 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:00:49.495759 30021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "9fa69fcac81740f3bd1e22254fc5fbd1"
format_stamp: "Formatted at 2026-08-06 15:00:49 on dist-test-slave-68xk"
server_key: "76d3a5b30ab389bd3f32ccd61600fc97"
server_key_iv: "a08964f2fcd8223bb75c8c055c2c562f"
server_key_version: "encryptionkey@0"
I20260806 15:00:49.495837 30021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:00:49.511698 30021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:00:49.513424 30021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:00:49.513578 30021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:00:49.517874 30021 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:35355
I20260806 15:00:49.517907 30121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:35355 every 8 connection(s)
I20260806 15:00:49.518254 30021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:00:49.519151 30124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:00:49.520666 30124 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35355 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:00:49.525815 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 30021
I20260806 15:00:49.525972 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:00:49.526180 11297 external_mini_cluster.cc:1428] Setting key 5cf98f992099a3971518e6fc3c2ad6bd
Aug 06 15:00:49 dist-test-slave-68xk krb5kdc[30001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028449, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:00:49.530057 30128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.521349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45404 (local address 127.11.8.126:35355)
0806 15:00:49.521513 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:49.521517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:49.521575 (+ 58us) server_negotiation.cc:407] Connection header received
0806 15:00:49.522080 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:49.522082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:49.522098 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:49.522180 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:49.522746 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.523554 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.524373 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.524685 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.524762 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:49.526822 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:49.526830 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:49.526831 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:49.526839 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:49.528538 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.529015 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.529017 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.529018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.529095 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.529340 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.529341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.529341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.529535 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:49.529614 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:49.529818 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 15:00:49.529908 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
I20260806 15:00:49.530200 30127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.521308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:35355 (local address 127.0.0.1:45404)
0806 15:00:49.521458 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:49.521498 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:49.521932 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:49.522202 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:49.522204 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:49.522512 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:00:49.522664 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:49.522669 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.523633 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.523634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:49.524303 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:49.524305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.524757 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.524758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:49.524810 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:49.525223 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:00:49.525247 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:00:49.526737 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:00:49.528630 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:49.528637 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:49.528639 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:49.528929 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:49.529161 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:49.529162 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:49.529163 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:49.529273 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:49.529604 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:00:49.529610 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:00:49.529682 (+ 72us) client_negotiation.cc:769] Sending connection context
0806 15:00:49.529787 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 15:00:49.530011 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":64,"thread_start_us":78,"threads_started":1}
I20260806 15:00:49.530923 30124 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.125" port: 32827 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:00:49.531258 30127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.531138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:32827 (local address 127.0.0.1:40174)
0806 15:00:49.531183 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:49.531233 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.125:32827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 15:00:49.531401 30051 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.8.125:32827: connect: Connection refused (error 111)
I20260806 15:00:49.531800 30127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.531695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:32827 (local address 127.0.0.1:40178)
0806 15:00:49.531740 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:49.531771 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.125:32827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 15:00:49.531955 30124 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.125:32827: Network error: Client connection negotiation failed: client connection to 127.11.8.125:32827: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.11.8.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.125.keytab.
Entry for principal kudu/127.11.8.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.125.keytab.
I20260806 15:00:49.554145 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.125:32827
--webserver_interface=127.11.8.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.125.keytab
--principal=kudu/127.11.8.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:39303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:35355,127.11.8.125:32827,127.11.8.124:36559
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:00:49.581212 30124 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.125" port: 32827 } attempt: 1
I20260806 15:00:49.581876 30127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.581724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:32827 (local address 127.0.0.1:40190)
0806 15:00:49.581790 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:49.581840 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.125:32827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 15:00:49.582321 30127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.582223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:32827 (local address 127.0.0.1:40200)
0806 15:00:49.582270 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:49.582301 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.125:32827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 15:00:49.582474 30124 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.125:32827: Network error: Client connection negotiation failed: client connection to 127.11.8.125:32827: connect: Connection refused (error 111)
W20260806 15:00:49.638597 30133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:00:49.638792 30133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:00:49.638814 30133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:00:49.639955 30133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:00:49.639999 30133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:00:49.640013 30133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:00:49.640065 30133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:00:49.640084 30133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:00:49.641546 30133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:39303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:35355,127.11.8.125:32827,127.11.8.124:36559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.125.keytab
--principal=kudu/127.11.8.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.125:32827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.8.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:00:49.641799 30133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:00:49.642025 30133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:00:49.647311 30143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:49.647329 30142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:49.647311 30145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:49.647707 30133 server_base.cc:1034] running on GCE node
I20260806 15:00:49.647953 30133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:00:49.648244 30133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:00:49.649405 30133 hybrid_clock.cc:648] HybridClock initialized: now 1786028449649396 us; error 35 us; skew 500 ppm
I20260806 15:00:49.650640 30124 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.125" port: 32827 } attempt: 2
I20260806 15:00:49.651044 30127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.650914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:32827 (local address 127.0.0.1:40206)
0806 15:00:49.650973 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:49.651007 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.125:32827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 15:00:49.651360 30127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.651271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:32827 (local address 127.0.0.1:40210)
0806 15:00:49.651312 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:49.651337 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.125:32827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 15:00:49.651495 30124 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.125:32827: Network error: Client connection negotiation failed: client connection to 127.11.8.125:32827: connect: Connection refused (error 111)
Aug 06 15:00:49 dist-test-slave-68xk krb5kdc[30001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028449, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:00:49.652082 30133 init.cc:377] Logged in from keytab as kudu/127.11.8.125@KRBTEST.COM (short username kudu)
I20260806 15:00:49.652515 30133 webserver.cc:460] Webserver started at http://127.11.8.125:41355/ using document root <none> and password file <none>
I20260806 15:00:49.652737 30133 fs_manager.cc:359] Metadata directory not provided
I20260806 15:00:49.652805 30133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:00:49.652953 30133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:00:49.653999 30133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-1/data/instance:
uuid: "e0cccf4b925945bd96a17305b9a16d38"
format_stamp: "Formatted at 2026-08-06 15:00:49 on dist-test-slave-68xk"
server_key: "bb372184a11b1f1697e706b2a6dff0fb"
server_key_iv: "ba4eacf526a92818f92f8c59552d9cc9"
server_key_version: "encryptionkey@0"
I20260806 15:00:49.654368 30133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-1/wal/instance:
uuid: "e0cccf4b925945bd96a17305b9a16d38"
format_stamp: "Formatted at 2026-08-06 15:00:49 on dist-test-slave-68xk"
server_key: "bb372184a11b1f1697e706b2a6dff0fb"
server_key_iv: "ba4eacf526a92818f92f8c59552d9cc9"
server_key_version: "encryptionkey@0"
I20260806 15:00:49.655716 30133 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:00:49.656416 30153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:49.656581 30133 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:00:49.656677 30133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-1/wal
uuid: "e0cccf4b925945bd96a17305b9a16d38"
format_stamp: "Formatted at 2026-08-06 15:00:49 on dist-test-slave-68xk"
server_key: "bb372184a11b1f1697e706b2a6dff0fb"
server_key_iv: "ba4eacf526a92818f92f8c59552d9cc9"
server_key_version: "encryptionkey@0"
I20260806 15:00:49.656760 30133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:00:49.677000 30133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:00:49.679056 30133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:00:49.679219 30133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:00:49.683315 30133 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.125:32827
I20260806 15:00:49.683351 30227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.125:32827 every 8 connection(s)
I20260806 15:00:49.683709 30133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-1/data/info.pb
I20260806 15:00:49.684573 30229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:00:49.686177 30229 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35355 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:00:49.690726 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 30133
I20260806 15:00:49.690855 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-1/wal/instance
I20260806 15:00:49.691057 11297 external_mini_cluster.cc:1428] Setting key 911d0bae8b31353cbdcd2c988cf5dad1
Aug 06 15:00:49 dist-test-slave-68xk krb5kdc[30001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028449, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.125@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:00:49.695153 30128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.686921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45412 (local address 127.11.8.126:35355)
0806 15:00:49.687002 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:49.687005 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:49.687202 (+ 197us) server_negotiation.cc:407] Connection header received
0806 15:00:49.687651 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:49.687652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:49.687665 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:49.687709 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:49.688296 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.688867 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.689792 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.689991 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.690046 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:49.692238 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:49.692240 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:49.692241 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:49.692249 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:49.693754 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.694166 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.694168 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.694168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.694230 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.694509 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.694510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.694510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.694626 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:49.694697 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:49.694970 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 15:00:49.695057 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 15:00:49.695384 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.686886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:35355 (local address 127.0.0.1:45412)
0806 15:00:49.687080 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:49.687118 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:49.687557 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:49.687794 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:49.687797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:49.688037 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:00:49.688223 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:49.688227 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.688949 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.688950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:49.689708 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:49.689711 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.690062 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.690063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:49.690161 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:49.690665 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:00:49.690674 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:00:49.692149 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:00:49.693840 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:49.693847 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:49.693849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:49.694087 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:49.694297 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:49.694298 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:49.694298 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:49.694443 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:49.694701 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:00:49.694708 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:00:49.694843 (+ 135us) client_negotiation.cc:769] Sending connection context
0806 15:00:49.694949 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 15:00:49.695143 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":98,"threads_started":1}
I20260806 15:00:49.695892 30229 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.125" port: 32827 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.8.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.124@KRBTEST.COM" created.
Aug 06 15:00:49 dist-test-slave-68xk krb5kdc[30001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028449, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.125@KRBTEST.COM for kudu/127.11.8.125@KRBTEST.COM
I20260806 15:00:49.706352 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.696177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:32827 (local address 127.0.0.1:40222)
0806 15:00:49.696239 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:49.696252 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:49.696304 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:49.696506 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:49.696507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:49.696674 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:00:49.696735 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:49.696737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.697538 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.697540 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:49.698442 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:49.698445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.698905 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.698907 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:49.698989 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:49.699372 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:00:49.699378 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:00:49.703154 (+ 3776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:00:49.705268 (+ 2114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:49.705269 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:49.705269 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:49.705431 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:49.705663 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:49.705664 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:49.705664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:49.705734 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:49.706032 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:00:49.706033 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:00:49.706071 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 15:00:49.706138 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 15:00:49.706242 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 15:00:49.706478 30235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.696214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40222 (local address 127.11.8.125:32827)
0806 15:00:49.696361 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:49.696366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:49.696384 (+ 18us) server_negotiation.cc:407] Connection header received
0806 15:00:49.696410 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:49.696412 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:49.696422 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:49.696483 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:49.696808 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.697454 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.698533 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.698824 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.698881 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:49.703263 (+ 4382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:49.703277 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:49.703277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:49.703283 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:49.705164 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.705516 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.705519 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.705519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.705596 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.705808 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.705809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.705809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.705961 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:49.706055 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:49.706166 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 15:00:49.706321 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":120,"thread_start_us":95,"threads_started":1}
I20260806 15:00:49.706985 30229 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36559 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:00:49.707345 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.707236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36559 (local address 127.0.0.1:40888)
0806 15:00:49.707282 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:49.707326 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 15:00:49.707474 30156 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36559: connect: Connection refused (error 111)
I20260806 15:00:49.707894 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.707803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36559 (local address 127.0.0.1:40900)
0806 15:00:49.707846 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:49.707871 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 15:00:49.707996 30229 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.124:36559: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36559: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.124.keytab.
Entry for principal kudu/127.11.8.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.124.keytab.
I20260806 15:00:49.719499 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.124:36559
--webserver_interface=127.11.8.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.124.keytab
--principal=kudu/127.11.8.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:39303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:35355,127.11.8.125:32827,127.11.8.124:36559
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:00:49.742642 30124 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.125" port: 32827 } attempt: 3
Aug 06 15:00:49 dist-test-slave-68xk krb5kdc[30001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028449, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for kudu/127.11.8.125@KRBTEST.COM
I20260806 15:00:49.750373 30127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.743016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:32827 (local address 127.0.0.1:40230)
0806 15:00:49.743071 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:49.743087 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:49.743148 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:49.743294 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:49.743294 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:49.743455 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:00:49.743519 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:49.743521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.744629 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.744631 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:49.745444 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:49.745447 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.745834 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.745835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:49.745915 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:49.746348 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:00:49.746353 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:00:49.747730 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:00:49.749345 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:49.749346 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:49.749347 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:49.749522 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:49.749731 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:49.749732 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:49.749732 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:49.749791 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:49.750037 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:00:49.750038 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:00:49.750086 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 15:00:49.750160 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 15:00:49.750267 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 15:00:49.750373 30235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.743089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40230 (local address 127.11.8.125:32827)
0806 15:00:49.743156 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:49.743159 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:49.743170 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:00:49.743211 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:49.743211 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:49.743222 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:49.743287 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:49.743609 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.744467 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.745547 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.745765 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.745820 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:49.747835 (+ 2015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:49.747837 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:49.747837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:49.747843 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:49.749246 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.749593 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.749595 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.749595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.749654 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.749856 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.749857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.749857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.749966 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:49.750035 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:49.750198 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 15:00:49.750282 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 15:00:49.750967 30124 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36559 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:00:49.751300 30127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.751197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36559 (local address 127.0.0.1:40902)
0806 15:00:49.751248 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:49.751276 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 15:00:49.751609 30127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.751555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36559 (local address 127.0.0.1:40906)
0806 15:00:49.751577 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:49.751597 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260806 15:00:49.751701 30124 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.124:36559: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36559: connect: Connection refused (error 111)
I20260806 15:00:49.757193 30229 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36559 } attempt: 1
I20260806 15:00:49.757530 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.757429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36559 (local address 127.0.0.1:40916)
0806 15:00:49.757478 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:49.757509 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 15:00:49.757831 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.757744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36559 (local address 127.0.0.1:40930)
0806 15:00:49.757783 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:49.757808 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 15:00:49.757915 30229 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.124:36559: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36559: connect: Connection refused (error 111)
I20260806 15:00:49.782859 30124 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36559 } attempt: 1
I20260806 15:00:49.783394 30127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.783256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36559 (local address 127.0.0.1:40932)
0806 15:00:49.783323 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:49.783365 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 15:00:49.783784 30127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.783691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36559 (local address 127.0.0.1:40938)
0806 15:00:49.783727 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:49.783759 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260806 15:00:49.783903 30124 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.124:36559: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36559: connect: Connection refused (error 111)
W20260806 15:00:49.804303 30238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:00:49.804497 30238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:00:49.804519 30238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:00:49.805596 30238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:00:49.805632 30238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:00:49.805646 30238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:00:49.805660 30238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:00:49.805671 30238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:00:49.807090 30238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:39303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:35355,127.11.8.125:32827,127.11.8.124:36559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.124.keytab
--principal=kudu/127.11.8.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.124:36559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.8.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:00:49.807289 30238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:00:49.807457 30238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:00:49.812256 30250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:49.812278 30246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:49.812264 30245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:49.812485 30238 server_base.cc:1034] running on GCE node
I20260806 15:00:49.812762 30238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:00:49.813052 30238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:00:49.814201 30238 hybrid_clock.cc:648] HybridClock initialized: now 1786028449814194 us; error 38 us; skew 500 ppm
Aug 06 15:00:49 dist-test-slave-68xk krb5kdc[30001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028449, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:00:49.816676 30238 init.cc:377] Logged in from keytab as kudu/127.11.8.124@KRBTEST.COM (short username kudu)
I20260806 15:00:49.817057 30238 webserver.cc:460] Webserver started at http://127.11.8.124:39919/ using document root <none> and password file <none>
I20260806 15:00:49.817307 30238 fs_manager.cc:359] Metadata directory not provided
I20260806 15:00:49.817378 30238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:00:49.817519 30238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:00:49.818635 30238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-2/data/instance:
uuid: "931ede28d6a740079296919102e8826e"
format_stamp: "Formatted at 2026-08-06 15:00:49 on dist-test-slave-68xk"
server_key: "79abc3c4039fbe41fba60dea6cbf049a"
server_key_iv: "be9cee87b1a0c57183c00c1860982535"
server_key_version: "encryptionkey@0"
I20260806 15:00:49.818992 30238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-2/wal/instance:
uuid: "931ede28d6a740079296919102e8826e"
format_stamp: "Formatted at 2026-08-06 15:00:49 on dist-test-slave-68xk"
server_key: "79abc3c4039fbe41fba60dea6cbf049a"
server_key_iv: "be9cee87b1a0c57183c00c1860982535"
server_key_version: "encryptionkey@0"
I20260806 15:00:49.820302 30238 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:00:49.820979 30260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:49.821143 30238 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:00:49.821256 30238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-2/wal
uuid: "931ede28d6a740079296919102e8826e"
format_stamp: "Formatted at 2026-08-06 15:00:49 on dist-test-slave-68xk"
server_key: "79abc3c4039fbe41fba60dea6cbf049a"
server_key_iv: "be9cee87b1a0c57183c00c1860982535"
server_key_version: "encryptionkey@0"
I20260806 15:00:49.821326 30238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:00:49.826051 30229 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36559 } attempt: 2
I20260806 15:00:49.826475 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.826314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36559 (local address 127.0.0.1:40954)
0806 15:00:49.826371 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:49.826439 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 15:00:49.826861 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.826767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36559 (local address 127.0.0.1:40958)
0806 15:00:49.826813 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:49.826838 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 15:00:49.826958 30229 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.124:36559: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36559: connect: Connection refused (error 111)
I20260806 15:00:49.842002 30238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:00:49.844195 30238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:00:49.844321 30238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:00:49.848412 30238 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.124:36559
I20260806 15:00:49.848456 30331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.124:36559 every 8 connection(s)
I20260806 15:00:49.848830 30238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-2/data/info.pb
I20260806 15:00:49.849740 30332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:00:49.851325 30332 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35355 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:00:49.854534 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 30238
I20260806 15:00:49.854658 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786028287744255-11297-0/minicluster-data/master-2/wal/instance
I20260806 15:00:49.854871 11297 external_mini_cluster.cc:1428] Setting key 5381e9ee29b5946bd18c27c046952eb0
Aug 06 15:00:49 dist-test-slave-68xk krb5kdc[30001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028449, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.124@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:00:49.859063 30124 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36559 } attempt: 2
Aug 06 15:00:49 dist-test-slave-68xk krb5kdc[30001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.124@KRBTEST.COM
I20260806 15:00:49.861526 30335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.852014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:35355 (local address 127.0.0.1:45424)
0806 15:00:49.852176 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:49.852218 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:49.852667 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:49.852902 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:49.852904 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:49.853185 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:00:49.853382 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:49.853386 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.854010 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.854011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:49.855013 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:49.855016 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.855526 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.855528 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:49.855712 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:49.856196 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:00:49.856206 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:00:49.857771 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:00:49.859859 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:49.859867 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:49.859869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:49.860203 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:49.860442 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:49.860443 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:49.860443 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:49.860582 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:49.860866 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:00:49.860874 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:00:49.861042 (+ 168us) client_negotiation.cc:769] Sending connection context
0806 15:00:49.861175 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 15:00:49.861337 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":79,"threads_started":1}
I20260806 15:00:49.861840 30128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.852054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45424 (local address 127.11.8.126:35355)
0806 15:00:49.852115 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:49.852117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:49.852298 (+ 181us) server_negotiation.cc:407] Connection header received
0806 15:00:49.852791 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:49.852793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:49.852802 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:49.852847 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:49.853453 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.853925 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.855094 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.855423 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.855499 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:49.857858 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:49.857860 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:49.857860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:49.857868 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:49.859778 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.860283 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.860285 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.860285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.860352 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.860658 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.860659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.860659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.860799 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:49.860870 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:49.861668 (+ 798us) server_negotiation.cc:299] Negotiation successful
0806 15:00:49.861756 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 15:00:49.862571 30332 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.125" port: 32827 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 15:00:49 dist-test-slave-68xk krb5kdc[30001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028449, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for kudu/127.11.8.124@KRBTEST.COM
I20260806 15:00:49.864585 30339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.855266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40970 (local address 127.11.8.124:36559)
0806 15:00:49.855423 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:49.855429 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:49.855455 (+ 26us) server_negotiation.cc:407] Connection header received
0806 15:00:49.855526 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:49.855530 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:49.855551 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:49.855618 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:49.856398 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.857164 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.857956 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.858379 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.858478 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:49.861259 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:49.861266 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:49.861267 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:49.861274 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:49.863257 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.863636 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.863639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.863639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.863696 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.863884 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.863884 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.863885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.864031 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:49.864182 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:49.864391 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 15:00:49.864468 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":91,"threads_started":1}
Aug 06 15:00:49 dist-test-slave-68xk krb5kdc[30001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028449, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.124@KRBTEST.COM for kudu/127.11.8.125@KRBTEST.COM
I20260806 15:00:49.868350 30127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.859320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36559 (local address 127.0.0.1:40976)
0806 15:00:49.859366 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:49.859379 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:49.859428 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:49.860184 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:49.860186 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:49.860338 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:00:49.860396 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:49.860398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.861185 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.861186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:49.861953 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:49.861957 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.862488 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.862489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:49.862557 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:49.862984 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:00:49.862989 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:00:49.864344 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:00:49.867249 (+ 2905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:49.867250 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:49.867250 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:49.867466 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:49.867691 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:49.867691 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:49.867692 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:49.867750 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:49.868003 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:00:49.868004 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:00:49.868053 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 15:00:49.868138 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 15:00:49.868244 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 15:00:49.869107 30340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.859914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40976 (local address 127.11.8.124:36559)
0806 15:00:49.860037 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:49.860039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:49.860046 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:00:49.860065 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:49.860065 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:49.860077 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:49.860467 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:49.860498 (+ 31us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.861109 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.862048 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.862355 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.862448 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:49.864435 (+ 1987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:49.864436 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:49.864436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:49.864443 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:49.867158 (+ 2715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.867547 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.867549 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.867549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.867611 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.867818 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.867818 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.867819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.867929 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:49.867988 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:49.868210 (+ 222us) server_negotiation.cc:299] Negotiation successful
0806 15:00:49.868952 (+ 742us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":73,"threads_started":1}
I20260806 15:00:49.869990 30335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.862812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:32827 (local address 127.0.0.1:40238)
0806 15:00:49.862848 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:49.862858 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:49.862901 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:49.863091 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:49.863091 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:49.863295 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:00:49.863329 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:49.863331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.864208 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.864209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:49.864949 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:49.864952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.865377 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.865378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:49.865438 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:49.865837 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:00:49.865842 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:00:49.867185 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:00:49.868899 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:49.868900 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:49.868900 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:49.869098 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:49.869335 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:49.869335 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:49.869336 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:49.869396 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:49.869669 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:00:49.869670 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:00:49.869721 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 15:00:49.869809 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 15:00:49.869904 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260806 15:00:49.870010 30235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.862910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40238 (local address 127.11.8.125:32827)
0806 15:00:49.862952 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:49.862953 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:49.862961 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:00:49.862989 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:49.862990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:49.863001 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:49.863064 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:49.863404 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.864115 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.865043 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.865290 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.865376 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:49.867284 (+ 1908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:49.867285 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:49.867286 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:49.867294 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:49.868818 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.869188 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.869189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.869190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.869262 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.869449 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.869450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.869450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.869590 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:49.869657 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:49.869856 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 15:00:49.869932 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 15:00:49.870662 30332 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36559 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 15:00:49 dist-test-slave-68xk krb5kdc[30001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:00:49.871207 30124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fa69fcac81740f3bd1e22254fc5fbd1: Bootstrap starting.
I20260806 15:00:49.872009 30124 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fa69fcac81740f3bd1e22254fc5fbd1: Neither blocks nor log segments found. Creating new log.
I20260806 15:00:49.872362 30124 log.cc:824] T 00000000000000000000000000000000 P 9fa69fcac81740f3bd1e22254fc5fbd1: Log is configured to *not* fsync() on all Append() calls
I20260806 15:00:49.873243 30124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fa69fcac81740f3bd1e22254fc5fbd1: No bootstrap required, opened a new log
I20260806 15:00:49.874114 30128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.865700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45432 (local address 127.11.8.126:35355)
0806 15:00:49.865737 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:49.865738 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:49.865750 (+ 12us) server_negotiation.cc:407] Connection header received
0806 15:00:49.865838 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:49.865840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:49.865853 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:49.865889 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:49.866602 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.867277 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.868271 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.868553 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.868593 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:49.871294 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:49.871295 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:49.871295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:49.871302 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:49.872910 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.873310 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.873311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.873311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.873368 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.873592 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.873594 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.873594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.873722 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:49.873879 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:49.873947 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 15:00:49.874024 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 06 15:00:49 dist-test-slave-68xk krb5kdc[30001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028449, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.124@KRBTEST.COM for kudu/127.11.8.124@KRBTEST.COM
I20260806 15:00:49.875548 30124 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fa69fcac81740f3bd1e22254fc5fbd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fa69fcac81740f3bd1e22254fc5fbd1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35355 } } peers { permanent_uuid: "e0cccf4b925945bd96a17305b9a16d38" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 32827 } } peers { permanent_uuid: "931ede28d6a740079296919102e8826e" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36559 } }
I20260806 15:00:49.875695 30124 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fa69fcac81740f3bd1e22254fc5fbd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:00:49.875727 30124 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fa69fcac81740f3bd1e22254fc5fbd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fa69fcac81740f3bd1e22254fc5fbd1, State: Initialized, Role: FOLLOWER
I20260806 15:00:49.875836 30124 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fa69fcac81740f3bd1e22254fc5fbd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fa69fcac81740f3bd1e22254fc5fbd1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35355 } } peers { permanent_uuid: "e0cccf4b925945bd96a17305b9a16d38" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 32827 } } peers { permanent_uuid: "931ede28d6a740079296919102e8826e" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36559 } }
I20260806 15:00:49.876134 30124 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fa69fcac81740f3bd1e22254fc5fbd1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:00:49.876399 30342 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fa69fcac81740f3bd1e22254fc5fbd1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fa69fcac81740f3bd1e22254fc5fbd1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35355 } } peers { permanent_uuid: "e0cccf4b925945bd96a17305b9a16d38" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 32827 } } peers { permanent_uuid: "931ede28d6a740079296919102e8826e" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36559 } } }
I20260806 15:00:49.876507 30342 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fa69fcac81740f3bd1e22254fc5fbd1 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 15:00:49.877995 30358 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9fa69fcac81740f3bd1e22254fc5fbd1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:00:49.878057 30358 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:00:49.878121 30335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.870925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36559 (local address 127.0.0.1:40982)
0806 15:00:49.870967 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:49.870976 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:49.871011 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:49.871129 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:49.871130 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:49.871279 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:00:49.871313 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:49.871315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.872152 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.872153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:49.872790 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:49.872793 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.873156 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.873157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:49.873204 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:49.873589 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:00:49.873594 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:00:49.874936 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:00:49.877078 (+ 2142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:49.877079 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:49.877079 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:49.877295 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:49.877483 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:49.877484 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:49.877484 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:49.877532 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:49.877837 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:00:49.877837 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:00:49.877873 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 15:00:49.877935 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 15:00:49.878020 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 15:00:49.878695 30340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.870986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40982 (local address 127.11.8.124:36559)
0806 15:00:49.871020 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:49.871023 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:49.871034 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:00:49.871056 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:49.871057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:49.871066 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:49.871126 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:49.871373 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.872057 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.872870 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.873077 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.873155 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:49.875424 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:49.875425 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:49.875425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:49.875431 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:49.876989 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.877375 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.877376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.877377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.877421 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.877590 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.877591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.877591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.877735 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:49.877800 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:49.878528 (+ 728us) server_negotiation.cc:299] Negotiation successful
0806 15:00:49.878614 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Aug 06 15:00:49 dist-test-slave-68xk krb5kdc[30001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.125@KRBTEST.COM
I20260806 15:00:49.878921 30021 master_runner.cc:186] Error getting master registration for 127.11.8.125:32827: OK, instance_id { permanent_uuid: "e0cccf4b925945bd96a17305b9a16d38" instance_seqno: 1786028449682085 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 15:00:49.881047 30021 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 15:00:49.882283 30332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e: Bootstrap starting.
I20260806 15:00:49.882982 30332 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e: Neither blocks nor log segments found. Creating new log.
I20260806 15:00:49.883320 30332 log.cc:824] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e: Log is configured to *not* fsync() on all Append() calls
I20260806 15:00:49.884049 30235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.874768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40240 (local address 127.11.8.125:32827)
0806 15:00:49.874805 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:49.874807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:49.874852 (+ 45us) server_negotiation.cc:407] Connection header received
0806 15:00:49.874914 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:49.874915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:49.874925 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:49.874986 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:49.875423 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.876051 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.877366 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.877689 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.877736 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:49.881100 (+ 3364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:49.881101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:49.881102 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:49.881109 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:49.882839 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.883317 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.883318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.883319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.883379 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.883603 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.883604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.883604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.883717 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:49.883846 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:49.883931 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 15:00:49.883990 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:00:49.885371 11297 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:00:49.887037 30332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e: No bootstrap required, opened a new log
I20260806 15:00:49.888486 30332 raft_consensus.cc:348] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fa69fcac81740f3bd1e22254fc5fbd1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35355 } } peers { permanent_uuid: "e0cccf4b925945bd96a17305b9a16d38" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 32827 } } peers { permanent_uuid: "931ede28d6a740079296919102e8826e" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36559 } }
I20260806 15:00:49.888603 30332 raft_consensus.cc:374] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:00:49.888629 30332 raft_consensus.cc:729] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 931ede28d6a740079296919102e8826e, State: Initialized, Role: FOLLOWER
I20260806 15:00:49.888773 30332 consensus_queue.cc:260] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fa69fcac81740f3bd1e22254fc5fbd1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35355 } } peers { permanent_uuid: "e0cccf4b925945bd96a17305b9a16d38" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 32827 } } peers { permanent_uuid: "931ede28d6a740079296919102e8826e" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36559 } }
I20260806 15:00:49.889065 30374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fa69fcac81740f3bd1e22254fc5fbd1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35355 } } peers { permanent_uuid: "e0cccf4b925945bd96a17305b9a16d38" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 32827 } } peers { permanent_uuid: "931ede28d6a740079296919102e8826e" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36559 } } }
I20260806 15:00:49.889109 30332 sys_catalog.cc:564] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:00:49.890508 30374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:00:49.893275 30238 master_runner.cc:186] Error getting master registration for 127.11.8.125:32827: OK, instance_id { permanent_uuid: "e0cccf4b925945bd96a17305b9a16d38" instance_seqno: 1786028449682085 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 15:00:49.896498 30238 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 15:00:49.897006 30128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.886712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45442 (local address 127.11.8.126:35355)
0806 15:00:49.886752 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:49.886754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:49.889320 (+ 2566us) server_negotiation.cc:407] Connection header received
0806 15:00:49.889522 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:49.889523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:49.889535 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:49.889567 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:49.890297 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.891071 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.892184 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.892458 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.892528 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:49.894040 (+ 1512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:49.894041 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:49.894042 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:49.894049 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:49.895684 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.896065 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.896066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.896067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.896120 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.896317 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.896317 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.896318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.896437 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:49.896521 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:49.896839 (+ 318us) server_negotiation.cc:299] Negotiation successful
0806 15:00:49.896913 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 15:00:49.902280 30235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.889371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40254 (local address 127.11.8.125:32827)
0806 15:00:49.889413 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:49.889415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:49.890499 (+ 1084us) server_negotiation.cc:407] Connection header received
0806 15:00:49.890654 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:49.890655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:49.890666 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:49.890694 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:49.893245 (+ 2551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.893988 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.895631 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.895903 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.895947 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:49.897672 (+ 1725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:49.897673 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:49.897674 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:49.897681 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:49.900167 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.900560 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.900561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.900561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.900613 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.900807 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.900808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.900809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.900924 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:49.901569 (+ 645us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:49.902083 (+ 514us) server_negotiation.cc:299] Negotiation successful
0806 15:00:49.902170 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 15:00:49.903034 30396 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:00:49.903095 30396 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:00:49.903959 30340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.889610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40990 (local address 127.11.8.124:36559)
0806 15:00:49.889648 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:49.889650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:49.896856 (+ 7206us) server_negotiation.cc:407] Connection header received
0806 15:00:49.896958 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:49.896959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:49.896970 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:49.897001 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:49.898123 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.898869 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.899720 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.899922 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.900004 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:49.901084 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:49.901085 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:49.901085 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:49.901092 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:49.902800 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.903197 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.903198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.903199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.903256 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.903455 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.903456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.903456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.903566 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:49.903694 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:49.903789 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 15:00:49.903861 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 15:00:49.918114 30229 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36559 } attempt: 3
Aug 06 15:00:49 dist-test-slave-68xk krb5kdc[30001](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028449, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.125@KRBTEST.COM for kudu/127.11.8.124@KRBTEST.COM
I20260806 15:00:49.925035 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.918456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36559 (local address 127.0.0.1:40996)
0806 15:00:49.918510 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:49.918521 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:49.918563 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:49.918735 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:49.918736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:49.918892 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:00:49.918931 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:49.918933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.919537 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.919538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:49.920278 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:49.920281 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.920634 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.920634 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:49.920688 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:49.921119 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:00:49.921129 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:00:49.922308 (+ 1179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:00:49.923988 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:49.923989 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:49.923990 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:49.924169 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:49.924363 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:49.924363 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:49.924364 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:49.924425 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:49.924693 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:00:49.924694 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:00:49.924736 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 15:00:49.924823 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 15:00:49.924925 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 15:00:49.925060 30340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:49.918513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40996 (local address 127.11.8.124:36559)
0806 15:00:49.918590 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:49.918592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:49.918629 (+ 37us) server_negotiation.cc:407] Connection header received
0806 15:00:49.918652 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:49.918653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:49.918663 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:49.918730 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:49.919007 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.919451 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.920368 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:49.920559 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:49.920634 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:49.922395 (+ 1761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:49.922397 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:49.922397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:49.922405 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:49.923914 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.924240 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.924241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.924242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.924299 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:49.924499 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:49.924500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:49.924500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:49.924616 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:49.924688 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:49.924860 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 15:00:49.924955 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 15:00:49.926659 30229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0cccf4b925945bd96a17305b9a16d38: Bootstrap starting.
I20260806 15:00:49.927301 30229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0cccf4b925945bd96a17305b9a16d38: Neither blocks nor log segments found. Creating new log.
I20260806 15:00:49.927616 30229 log.cc:824] T 00000000000000000000000000000000 P e0cccf4b925945bd96a17305b9a16d38: Log is configured to *not* fsync() on all Append() calls
I20260806 15:00:49.928692 30229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0cccf4b925945bd96a17305b9a16d38: No bootstrap required, opened a new log
I20260806 15:00:49.930188 30229 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0cccf4b925945bd96a17305b9a16d38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fa69fcac81740f3bd1e22254fc5fbd1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35355 } } peers { permanent_uuid: "e0cccf4b925945bd96a17305b9a16d38" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 32827 } } peers { permanent_uuid: "931ede28d6a740079296919102e8826e" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36559 } }
I20260806 15:00:49.930315 30229 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0cccf4b925945bd96a17305b9a16d38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:00:49.930361 30229 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0cccf4b925945bd96a17305b9a16d38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0cccf4b925945bd96a17305b9a16d38, State: Initialized, Role: FOLLOWER
I20260806 15:00:49.930524 30229 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0cccf4b925945bd96a17305b9a16d38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fa69fcac81740f3bd1e22254fc5fbd1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35355 } } peers { permanent_uuid: "e0cccf4b925945bd96a17305b9a16d38" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 32827 } } peers { permanent_uuid: "931ede28d6a740079296919102e8826e" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36559 } }
I20260806 15:00:49.930806 30401 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0cccf4b925945bd96a17305b9a16d38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fa69fcac81740f3bd1e22254fc5fbd1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35355 } } peers { permanent_uuid: "e0cccf4b925945bd96a17305b9a16d38" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 32827 } } peers { permanent_uuid: "931ede28d6a740079296919102e8826e" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36559 } } }
I20260806 15:00:49.930871 30229 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0cccf4b925945bd96a17305b9a16d38 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:00:49.930946 30401 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0cccf4b925945bd96a17305b9a16d38 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 15:00:49.932756 30420 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e0cccf4b925945bd96a17305b9a16d38: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:00:49.932862 30420 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:00:49.934722 30133 master_runner.cc:263] No leader master found from master e0cccf4b925945bd96a17305b9a16d38
I20260806 15:00:49.934769 30133 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 15:00:50.014348 30055 tablet.cc:2369] T 00000000000000000000000000000000 P 9fa69fcac81740f3bd1e22254fc5fbd1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:00:50.019268 30374 raft_consensus.cc:482] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:00:50.019394 30374 raft_consensus.cc:504] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fa69fcac81740f3bd1e22254fc5fbd1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35355 } } peers { permanent_uuid: "e0cccf4b925945bd96a17305b9a16d38" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 32827 } } peers { permanent_uuid: "931ede28d6a740079296919102e8826e" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36559 } }
I20260806 15:00:50.019644 30374 leader_election.cc:288] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9fa69fcac81740f3bd1e22254fc5fbd1 (127.11.8.126:35355), e0cccf4b925945bd96a17305b9a16d38 (127.11.8.125:32827)
I20260806 15:00:50.020004 30078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "931ede28d6a740079296919102e8826e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fa69fcac81740f3bd1e22254fc5fbd1" is_pre_election: true
I20260806 15:00:50.020171 30078 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9fa69fcac81740f3bd1e22254fc5fbd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 931ede28d6a740079296919102e8826e in term 0.
I20260806 15:00:50.020604 30262 leader_election.cc:302] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 931ede28d6a740079296919102e8826e, 9fa69fcac81740f3bd1e22254fc5fbd1; no voters:
I20260806 15:00:50.020828 30374 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:00:50.020891 30374 raft_consensus.cc:482] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:00:50.020934 30374 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:00:50.021909 30374 raft_consensus.cc:504] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fa69fcac81740f3bd1e22254fc5fbd1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35355 } } peers { permanent_uuid: "e0cccf4b925945bd96a17305b9a16d38" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 32827 } } peers { permanent_uuid: "931ede28d6a740079296919102e8826e" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36559 } }
I20260806 15:00:50.022047 30374 leader_election.cc:288] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e [CANDIDATE]: Term 1 election: Requested vote from peers 9fa69fcac81740f3bd1e22254fc5fbd1 (127.11.8.126:35355), e0cccf4b925945bd96a17305b9a16d38 (127.11.8.125:32827)
I20260806 15:00:50.022389 30078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "931ede28d6a740079296919102e8826e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fa69fcac81740f3bd1e22254fc5fbd1"
I20260806 15:00:50.022511 30078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fa69fcac81740f3bd1e22254fc5fbd1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:00:50.023437 30078 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9fa69fcac81740f3bd1e22254fc5fbd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 931ede28d6a740079296919102e8826e in term 1.
I20260806 15:00:50.023680 30262 leader_election.cc:302] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 931ede28d6a740079296919102e8826e, 9fa69fcac81740f3bd1e22254fc5fbd1; no voters:
I20260806 15:00:50.024029 30195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "931ede28d6a740079296919102e8826e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cccf4b925945bd96a17305b9a16d38" is_pre_election: true
I20260806 15:00:50.024192 30195 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e0cccf4b925945bd96a17305b9a16d38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 931ede28d6a740079296919102e8826e in term 0.
I20260806 15:00:50.024052 30191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "931ede28d6a740079296919102e8826e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cccf4b925945bd96a17305b9a16d38"
I20260806 15:00:50.024335 30191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0cccf4b925945bd96a17305b9a16d38 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:00:50.025208 30191 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e0cccf4b925945bd96a17305b9a16d38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 931ede28d6a740079296919102e8826e in term 1.
I20260806 15:00:50.026401 30374 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:00:50.026598 30374 raft_consensus.cc:686] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e [term 1 LEADER]: Becoming Leader. State: Replica: 931ede28d6a740079296919102e8826e, State: Running, Role: LEADER
I20260806 15:00:50.026684 30374 consensus_queue.cc:237] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fa69fcac81740f3bd1e22254fc5fbd1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35355 } } peers { permanent_uuid: "e0cccf4b925945bd96a17305b9a16d38" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 32827 } } peers { permanent_uuid: "931ede28d6a740079296919102e8826e" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36559 } }
I20260806 15:00:50.027062 30374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 931ede28d6a740079296919102e8826e. Latest consensus state: current_term: 1 leader_uuid: "931ede28d6a740079296919102e8826e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fa69fcac81740f3bd1e22254fc5fbd1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35355 } } peers { permanent_uuid: "e0cccf4b925945bd96a17305b9a16d38" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 32827 } } peers { permanent_uuid: "931ede28d6a740079296919102e8826e" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36559 } } }
I20260806 15:00:50.027134 30374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e [sys.catalog]: This master's current role is: LEADER
I20260806 15:00:50.028005 30426 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:00:50.028246 30426 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:00:50.029793 30191 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e0cccf4b925945bd96a17305b9a16d38 [term 1 FOLLOWER]: Refusing update from remote peer 931ede28d6a740079296919102e8826e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:00:50.030179 30423 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cccf4b925945bd96a17305b9a16d38" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 32827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:00:50.031695 30401 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0cccf4b925945bd96a17305b9a16d38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 931ede28d6a740079296919102e8826e. Latest consensus state: current_term: 1 leader_uuid: "931ede28d6a740079296919102e8826e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fa69fcac81740f3bd1e22254fc5fbd1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35355 } } peers { permanent_uuid: "e0cccf4b925945bd96a17305b9a16d38" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 32827 } } peers { permanent_uuid: "931ede28d6a740079296919102e8826e" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36559 } } }
I20260806 15:00:50.031790 30401 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0cccf4b925945bd96a17305b9a16d38 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:00:50.032276 30078 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9fa69fcac81740f3bd1e22254fc5fbd1 [term 1 FOLLOWER]: Refusing update from remote peer 931ede28d6a740079296919102e8826e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:00:50.032580 30423 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fa69fcac81740f3bd1e22254fc5fbd1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:00:50.033797 30342 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fa69fcac81740f3bd1e22254fc5fbd1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 931ede28d6a740079296919102e8826e. Latest consensus state: current_term: 1 leader_uuid: "931ede28d6a740079296919102e8826e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fa69fcac81740f3bd1e22254fc5fbd1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35355 } } peers { permanent_uuid: "e0cccf4b925945bd96a17305b9a16d38" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 32827 } } peers { permanent_uuid: "931ede28d6a740079296919102e8826e" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36559 } } }
I20260806 15:00:50.033881 30342 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fa69fcac81740f3bd1e22254fc5fbd1 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:00:50.035130 30423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "931ede28d6a740079296919102e8826e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fa69fcac81740f3bd1e22254fc5fbd1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35355 } } peers { permanent_uuid: "e0cccf4b925945bd96a17305b9a16d38" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 32827 } } peers { permanent_uuid: "931ede28d6a740079296919102e8826e" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36559 } } }
I20260806 15:00:50.035213 30423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e [sys.catalog]: This master's current role is: LEADER
I20260806 15:00:50.035741 30401 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0cccf4b925945bd96a17305b9a16d38 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "931ede28d6a740079296919102e8826e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fa69fcac81740f3bd1e22254fc5fbd1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35355 } } peers { permanent_uuid: "e0cccf4b925945bd96a17305b9a16d38" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 32827 } } peers { permanent_uuid: "931ede28d6a740079296919102e8826e" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36559 } } }
I20260806 15:00:50.035838 30401 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0cccf4b925945bd96a17305b9a16d38 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:00:50.036162 30428 mvcc.cc:205] Tried to move back new op lower bound from 7315572531319812096 to 7315572531309867008. Current Snapshot: MvccSnapshot[applied={T|T < 7315572531319812096}]
I20260806 15:00:50.036315 30429 mvcc.cc:205] Tried to move back new op lower bound from 7315572531319812096 to 7315572531309867008. Current Snapshot: MvccSnapshot[applied={T|T < 7315572531319812096}]
I20260806 15:00:50.037189 30374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "931ede28d6a740079296919102e8826e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fa69fcac81740f3bd1e22254fc5fbd1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35355 } } peers { permanent_uuid: "e0cccf4b925945bd96a17305b9a16d38" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 32827 } } peers { permanent_uuid: "931ede28d6a740079296919102e8826e" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36559 } } }
I20260806 15:00:50.037261 30374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e [sys.catalog]: This master's current role is: LEADER
I20260806 15:00:50.037648 30342 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fa69fcac81740f3bd1e22254fc5fbd1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "931ede28d6a740079296919102e8826e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fa69fcac81740f3bd1e22254fc5fbd1" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35355 } } peers { permanent_uuid: "e0cccf4b925945bd96a17305b9a16d38" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 32827 } } peers { permanent_uuid: "931ede28d6a740079296919102e8826e" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36559 } } }
I20260806 15:00:50.037720 30342 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fa69fcac81740f3bd1e22254fc5fbd1 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:00:50.038393 30432 mvcc.cc:205] Tried to move back new op lower bound from 7315572531319812096 to 7315572531309867008. Current Snapshot: MvccSnapshot[applied={T|T < 7315572531319812096}]
I20260806 15:00:50.039384 30426 catalog_manager.cc:1347] Generated new cluster ID: 9ae5c5992d4446a09f7377db697b712b
I20260806 15:00:50.039430 30426 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:00:50.056263 30426 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:00:50.057107 30426 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:00:50.063637 30426 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 931ede28d6a740079296919102e8826e: Generated new TSK 0
I20260806 15:00:50.063879 30426 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:00:50.878515 30358 catalog_manager.cc:1259] Loaded cluster ID: 9ae5c5992d4446a09f7377db697b712b
I20260806 15:00:50.878598 30358 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9fa69fcac81740f3bd1e22254fc5fbd1: loading cluster ID for follower catalog manager: success
I20260806 15:00:50.879472 30358 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9fa69fcac81740f3bd1e22254fc5fbd1: acquiring CA information for follower catalog manager: success
I20260806 15:00:50.879750 30358 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9fa69fcac81740f3bd1e22254fc5fbd1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 15:00:50.933342 30420 catalog_manager.cc:1259] Loaded cluster ID: 9ae5c5992d4446a09f7377db697b712b
I20260806 15:00:50.933426 30420 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e0cccf4b925945bd96a17305b9a16d38: loading cluster ID for follower catalog manager: success
I20260806 15:00:50.934274 30420 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e0cccf4b925945bd96a17305b9a16d38: acquiring CA information for follower catalog manager: success
I20260806 15:00:50.934624 30420 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e0cccf4b925945bd96a17305b9a16d38: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 15:00:51.112012 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 30021
I20260806 15:00:51.123636 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 30133
I20260806 15:00:51.130396 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 30238
2026-08-06T15:00:51Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3075 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:00:51 dist-test-slave-68xk krb5kdc[30447](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:00:51 dist-test-slave-68xk krb5kdc[30447](info): set up 2 sockets
Aug 06 15:00:51 dist-test-slave-68xk krb5kdc[30447](info): commencing operation
krb5kdc: starting...
W20260806 15:00:52.345116 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 15:00:52 dist-test-slave-68xk krb5kdc[30447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:00:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:00:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 15:00:52.471489 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:38283
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:42845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:38283
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:00:52.555142 30464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:00:52.555364 30464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:00:52.555387 30464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:00:52.556504 30464 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:00:52.556543 30464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:00:52.556558 30464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:00:52.556571 30464 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:00:52.556583 30464 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:00:52.558012 30464 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:42845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:38283
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:38283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:00:52.558221 30464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:00:52.558400 30464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:00:52.563153 30471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:52.563145 30473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:52.563171 30476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:52.563602 30464 server_base.cc:1034] running on GCE node
I20260806 15:00:52.563850 30464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:00:52.564167 30464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:00:52.565331 30464 hybrid_clock.cc:648] HybridClock initialized: now 1786028452565312 us; error 37 us; skew 500 ppm
Aug 06 15:00:52 dist-test-slave-68xk krb5kdc[30447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028452, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:00:52.567924 30464 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 15:00:52.568323 30464 webserver.cc:460] Webserver started at http://127.11.8.126:45405/ using document root <none> and password file <none>
I20260806 15:00:52.568517 30464 fs_manager.cc:359] Metadata directory not provided
I20260806 15:00:52.568557 30464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:00:52.568646 30464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:00:52.569653 30464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "d155dd6f40994f8c8b783f910672a451"
format_stamp: "Formatted at 2026-08-06 15:00:52 on dist-test-slave-68xk"
server_key: "61fe5fbcb495859869ac6428b791e780"
server_key_iv: "ee57be609c4dbed2bf887474a2a00505"
server_key_version: "encryptionkey@0"
I20260806 15:00:52.569955 30464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "d155dd6f40994f8c8b783f910672a451"
format_stamp: "Formatted at 2026-08-06 15:00:52 on dist-test-slave-68xk"
server_key: "61fe5fbcb495859869ac6428b791e780"
server_key_iv: "ee57be609c4dbed2bf887474a2a00505"
server_key_version: "encryptionkey@0"
I20260806 15:00:52.571357 30464 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:00:52.572045 30485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:52.572224 30464 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:52.572315 30464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "d155dd6f40994f8c8b783f910672a451"
format_stamp: "Formatted at 2026-08-06 15:00:52 on dist-test-slave-68xk"
server_key: "61fe5fbcb495859869ac6428b791e780"
server_key_iv: "ee57be609c4dbed2bf887474a2a00505"
server_key_version: "encryptionkey@0"
I20260806 15:00:52.572389 30464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:00:52.583114 30464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:00:52.585400 30464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:00:52.585568 30464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:00:52.590209 30464 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:38283
I20260806 15:00:52.590277 30570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:38283 every 8 connection(s)
I20260806 15:00:52.590636 30464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:00:52.591568 30571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:00:52.593896 30571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d155dd6f40994f8c8b783f910672a451: Bootstrap starting.
I20260806 15:00:52.594570 30571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d155dd6f40994f8c8b783f910672a451: Neither blocks nor log segments found. Creating new log.
I20260806 15:00:52.594882 30571 log.cc:824] T 00000000000000000000000000000000 P d155dd6f40994f8c8b783f910672a451: Log is configured to *not* fsync() on all Append() calls
I20260806 15:00:52.595631 30571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d155dd6f40994f8c8b783f910672a451: No bootstrap required, opened a new log
I20260806 15:00:52.596881 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 30464
I20260806 15:00:52.596987 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:00:52.597221 11297 external_mini_cluster.cc:1428] Setting key 4bd475969ebfafb243864e029dbbcdaa
I20260806 15:00:52.597678 30571 raft_consensus.cc:348] T 00000000000000000000000000000000 P d155dd6f40994f8c8b783f910672a451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d155dd6f40994f8c8b783f910672a451" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38283 } }
I20260806 15:00:52.597887 30571 raft_consensus.cc:374] T 00000000000000000000000000000000 P d155dd6f40994f8c8b783f910672a451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:00:52.597987 30571 raft_consensus.cc:729] T 00000000000000000000000000000000 P d155dd6f40994f8c8b783f910672a451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d155dd6f40994f8c8b783f910672a451, State: Initialized, Role: FOLLOWER
I20260806 15:00:52.598131 30571 consensus_queue.cc:260] T 00000000000000000000000000000000 P d155dd6f40994f8c8b783f910672a451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d155dd6f40994f8c8b783f910672a451" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38283 } }
I20260806 15:00:52.598213 30571 raft_consensus.cc:388] T 00000000000000000000000000000000 P d155dd6f40994f8c8b783f910672a451 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:00:52.598235 30571 raft_consensus.cc:482] T 00000000000000000000000000000000 P d155dd6f40994f8c8b783f910672a451 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:00:52.598307 30571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d155dd6f40994f8c8b783f910672a451 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:00:52.599292 30571 raft_consensus.cc:504] T 00000000000000000000000000000000 P d155dd6f40994f8c8b783f910672a451 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d155dd6f40994f8c8b783f910672a451" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38283 } }
I20260806 15:00:52.599383 30571 leader_election.cc:302] T 00000000000000000000000000000000 P d155dd6f40994f8c8b783f910672a451 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d155dd6f40994f8c8b783f910672a451; no voters:
I20260806 15:00:52.599596 30571 leader_election.cc:288] T 00000000000000000000000000000000 P d155dd6f40994f8c8b783f910672a451 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:00:52.599627 30576 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d155dd6f40994f8c8b783f910672a451 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:00:52.599735 30576 raft_consensus.cc:686] T 00000000000000000000000000000000 P d155dd6f40994f8c8b783f910672a451 [term 1 LEADER]: Becoming Leader. State: Replica: d155dd6f40994f8c8b783f910672a451, State: Running, Role: LEADER
I20260806 15:00:52.599871 30571 sys_catalog.cc:564] T 00000000000000000000000000000000 P d155dd6f40994f8c8b783f910672a451 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:00:52.599891 30576 consensus_queue.cc:237] T 00000000000000000000000000000000 P d155dd6f40994f8c8b783f910672a451 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d155dd6f40994f8c8b783f910672a451" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38283 } }
I20260806 15:00:52.600299 30576 sys_catalog.cc:455] T 00000000000000000000000000000000 P d155dd6f40994f8c8b783f910672a451 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d155dd6f40994f8c8b783f910672a451. Latest consensus state: current_term: 1 leader_uuid: "d155dd6f40994f8c8b783f910672a451" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d155dd6f40994f8c8b783f910672a451" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38283 } } }
I20260806 15:00:52.600307 30577 sys_catalog.cc:455] T 00000000000000000000000000000000 P d155dd6f40994f8c8b783f910672a451 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d155dd6f40994f8c8b783f910672a451" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d155dd6f40994f8c8b783f910672a451" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38283 } } }
I20260806 15:00:52.600483 30576 sys_catalog.cc:458] T 00000000000000000000000000000000 P d155dd6f40994f8c8b783f910672a451 [sys.catalog]: This master's current role is: LEADER
I20260806 15:00:52.600666 30577 sys_catalog.cc:458] T 00000000000000000000000000000000 P d155dd6f40994f8c8b783f910672a451 [sys.catalog]: This master's current role is: LEADER
I20260806 15:00:52.600919 30590 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:00:52.601534 30590 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:00:52.603308 30590 catalog_manager.cc:1347] Generated new cluster ID: 66e0d77dbce748ae8bbaca680a71b8bd
I20260806 15:00:52.603367 30590 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 15:00:52 dist-test-slave-68xk krb5kdc[30447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:00:52.608064 30575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:52.597743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35634 (local address 127.11.8.126:38283)
0806 15:00:52.597933 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:52.597943 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:52.597982 (+ 39us) server_negotiation.cc:407] Connection header received
0806 15:00:52.598117 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:52.598125 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:52.598222 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:52.598368 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:52.599116 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:52.600343 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:52.601482 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:52.602065 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:52.602193 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:52.604723 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:52.604727 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:52.604727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:52.604745 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:52.606393 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:52.606792 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:52.606795 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:52.606795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:52.606915 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:52.607133 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:52.607134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:52.607134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:52.607360 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:52.607529 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:52.607597 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 15:00:52.607755 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":71,"threads_started":1}
I20260806 15:00:52.609103 11297 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:00:52.616508 30590 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:00:52.617028 30590 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:00:52.626253 30590 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d155dd6f40994f8c8b783f910672a451: Generated new TSK 0
I20260806 15:00:52.626478 30590 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:00:52.663220 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 30464
2026-08-06T15:00:52Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1535 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:00:52 dist-test-slave-68xk krb5kdc[30611](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:00:52 dist-test-slave-68xk krb5kdc[30611](info): set up 2 sockets
Aug 06 15:00:52 dist-test-slave-68xk krb5kdc[30611](info): commencing operation
krb5kdc: starting...
W20260806 15:00:53.871865 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 15:00:53 dist-test-slave-68xk krb5kdc[30611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:00:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:00:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 15:00:53.996563 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:38995
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:37615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:38995
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:00:54.081230 30636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:00:54.081450 30636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:00:54.081475 30636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:00:54.082780 30636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:00:54.082824 30636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:00:54.082840 30636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:00:54.082894 30636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:00:54.082913 30636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:00:54.084553 30636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:37615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:38995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:38995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:00:54.084813 30636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:00:54.085075 30636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:00:54.090206 30645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:54.090193 30647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:54.090342 30644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:54.090667 30636 server_base.cc:1034] running on GCE node
I20260806 15:00:54.091173 30636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:00:54.091451 30636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:00:54.092618 30636 hybrid_clock.cc:648] HybridClock initialized: now 1786028454092613 us; error 44 us; skew 500 ppm
Aug 06 15:00:54 dist-test-slave-68xk krb5kdc[30611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028454, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:00:54.095266 30636 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 15:00:54.095738 30636 webserver.cc:460] Webserver started at http://127.11.8.126:32903/ using document root <none> and password file <none>
I20260806 15:00:54.095953 30636 fs_manager.cc:359] Metadata directory not provided
I20260806 15:00:54.095993 30636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:00:54.096083 30636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:00:54.097088 30636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "2f76c3cc03194bdcb237406279446e30"
format_stamp: "Formatted at 2026-08-06 15:00:54 on dist-test-slave-68xk"
server_key: "68b434bd754da26e7bed7757697f78fa"
server_key_iv: "e1ece243453028139ea624c9e6a98846"
server_key_version: "encryptionkey@0"
I20260806 15:00:54.097420 30636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "2f76c3cc03194bdcb237406279446e30"
format_stamp: "Formatted at 2026-08-06 15:00:54 on dist-test-slave-68xk"
server_key: "68b434bd754da26e7bed7757697f78fa"
server_key_iv: "e1ece243453028139ea624c9e6a98846"
server_key_version: "encryptionkey@0"
I20260806 15:00:54.098851 30636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:00:54.099639 30656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:54.099830 30636 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:00:54.099920 30636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "2f76c3cc03194bdcb237406279446e30"
format_stamp: "Formatted at 2026-08-06 15:00:54 on dist-test-slave-68xk"
server_key: "68b434bd754da26e7bed7757697f78fa"
server_key_iv: "e1ece243453028139ea624c9e6a98846"
server_key_version: "encryptionkey@0"
I20260806 15:00:54.099978 30636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:00:54.137938 30636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:00:54.140359 30636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:00:54.140478 30636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:00:54.145088 30636 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:38995
I20260806 15:00:54.145191 30738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:38995 every 8 connection(s)
I20260806 15:00:54.145534 30636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:00:54.147401 30739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:00:54.149730 30739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30: Bootstrap starting.
I20260806 15:00:54.150357 30739 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30: Neither blocks nor log segments found. Creating new log.
I20260806 15:00:54.150683 30739 log.cc:824] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30: Log is configured to *not* fsync() on all Append() calls
I20260806 15:00:54.151422 30739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30: No bootstrap required, opened a new log
I20260806 15:00:54.151836 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 30636
I20260806 15:00:54.151966 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:00:54.152196 11297 external_mini_cluster.cc:1428] Setting key 429e1e975f67884451c75d7d435552d0
I20260806 15:00:54.153584 30739 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f76c3cc03194bdcb237406279446e30" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38995 } }
I20260806 15:00:54.153746 30739 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:00:54.153811 30739 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f76c3cc03194bdcb237406279446e30, State: Initialized, Role: FOLLOWER
I20260806 15:00:54.153983 30739 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f76c3cc03194bdcb237406279446e30" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38995 } }
I20260806 15:00:54.154072 30739 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:00:54.154124 30739 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:00:54.154174 30739 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:00:54.155181 30739 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f76c3cc03194bdcb237406279446e30" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38995 } }
I20260806 15:00:54.155304 30739 leader_election.cc:302] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f76c3cc03194bdcb237406279446e30; no voters:
I20260806 15:00:54.155483 30739 leader_election.cc:288] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:00:54.155527 30744 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:00:54.155709 30744 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [term 1 LEADER]: Becoming Leader. State: Replica: 2f76c3cc03194bdcb237406279446e30, State: Running, Role: LEADER
I20260806 15:00:54.155746 30739 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:00:54.155879 30744 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f76c3cc03194bdcb237406279446e30" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38995 } }
I20260806 15:00:54.156243 30744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f76c3cc03194bdcb237406279446e30. Latest consensus state: current_term: 1 leader_uuid: "2f76c3cc03194bdcb237406279446e30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f76c3cc03194bdcb237406279446e30" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38995 } } }
I20260806 15:00:54.156375 30744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [sys.catalog]: This master's current role is: LEADER
I20260806 15:00:54.156347 30745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f76c3cc03194bdcb237406279446e30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f76c3cc03194bdcb237406279446e30" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38995 } } }
I20260806 15:00:54.156471 30745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [sys.catalog]: This master's current role is: LEADER
I20260806 15:00:54.156775 30754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:00:54.157305 30754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 15:00:54 dist-test-slave-68xk krb5kdc[30611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:00:54.162525 30754 catalog_manager.cc:1347] Generated new cluster ID: f2ad979d149846c9a59bb665feff6281
I20260806 15:00:54.162590 30754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:00:54.167586 30743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:54.152734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47694 (local address 127.11.8.126:38995)
0806 15:00:54.152937 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:54.152947 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:54.152977 (+ 30us) server_negotiation.cc:407] Connection header received
0806 15:00:54.153564 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:54.153572 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:54.153623 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:54.153783 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:54.154554 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.155562 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.156709 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.157331 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.157798 (+ 467us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:54.160152 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:54.160155 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:54.160156 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:54.160166 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:54.161864 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:54.162278 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:54.162281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:54.162281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:54.162405 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:54.166692 (+ 4287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:54.166693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:54.166693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:54.166959 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:54.167185 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:54.167270 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 15:00:54.167419 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":90,"threads_started":1}
I20260806 15:00:54.168709 30754 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:00:54.169193 30754 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:00:54.174330 30754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30: Generated new TSK 0
I20260806 15:00:54.174562 30754 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
I20260806 15:00:54.207480 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:0
--local_ip_for_outbound_sockets=127.11.8.65
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:38995
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:37615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:00:54.321630 30770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:00:54.321825 30770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:00:54.321848 30770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:00:54.323021 30770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:00:54.323063 30770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:00:54.323091 30770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 15:00:54.324550 30770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:37615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:38995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:00:54.324791 30770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:00:54.325040 30770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:00:54.330121 30779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:54.330129 30780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:54.330251 30770 server_base.cc:1034] running on GCE node
W20260806 15:00:54.330449 30782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:54.330662 30770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:00:54.330914 30770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:00:54.332082 30770 hybrid_clock.cc:648] HybridClock initialized: now 1786028454332059 us; error 39 us; skew 500 ppm
Aug 06 15:00:54 dist-test-slave-68xk krb5kdc[30611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028454, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:00:54.334558 30770 init.cc:377] Logged in from keytab as kudu/127.11.8.65@KRBTEST.COM (short username kudu)
I20260806 15:00:54.335011 30770 webserver.cc:460] Webserver started at http://127.11.8.65:32789/ using document root <none> and password file <none>
I20260806 15:00:54.335263 30770 fs_manager.cc:359] Metadata directory not provided
I20260806 15:00:54.335331 30770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:00:54.335469 30770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:00:54.336509 30770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/data/instance:
uuid: "39babaadc45f4e808c7cbb9b96805b7c"
format_stamp: "Formatted at 2026-08-06 15:00:54 on dist-test-slave-68xk"
server_key: "a6b677c8b0f284677a2e52a57b42da2d"
server_key_iv: "3b66dac8d3c8158f34911f5d3118c833"
server_key_version: "encryptionkey@0"
I20260806 15:00:54.336885 30770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance:
uuid: "39babaadc45f4e808c7cbb9b96805b7c"
format_stamp: "Formatted at 2026-08-06 15:00:54 on dist-test-slave-68xk"
server_key: "a6b677c8b0f284677a2e52a57b42da2d"
server_key_iv: "3b66dac8d3c8158f34911f5d3118c833"
server_key_version: "encryptionkey@0"
I20260806 15:00:54.338292 30770 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:00:54.339114 30793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:54.339289 30770 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:00:54.339388 30770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "39babaadc45f4e808c7cbb9b96805b7c"
format_stamp: "Formatted at 2026-08-06 15:00:54 on dist-test-slave-68xk"
server_key: "a6b677c8b0f284677a2e52a57b42da2d"
server_key_iv: "3b66dac8d3c8158f34911f5d3118c833"
server_key_version: "encryptionkey@0"
I20260806 15:00:54.339466 30770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:00:54.356756 30770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:00:54.358771 30770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:00:54.358933 30770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:00:54.359154 30770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:00:54.359462 30770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:00:54.359515 30770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:54.359549 30770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:00:54.359577 30770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:54.365643 30770 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:34479
I20260806 15:00:54.365724 30957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:34479 every 8 connection(s)
I20260806 15:00:54.366030 30770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
Aug 06 15:00:54 dist-test-slave-68xk krb5kdc[30611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028454, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:00:54.374058 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 30770
I20260806 15:00:54.374194 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance
I20260806 15:00:54.374442 11297 external_mini_cluster.cc:1428] Setting key 8c9c5de29ad8ae4d5004788f5168f007
I20260806 15:00:54.375488 30743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:54.366798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:40099 (local address 127.11.8.126:38995)
0806 15:00:54.366857 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:54.366859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:54.367045 (+ 186us) server_negotiation.cc:407] Connection header received
0806 15:00:54.367527 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:54.367528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:54.367541 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:54.367598 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:54.368520 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.369080 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.370025 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.370254 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.370326 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:54.372379 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:54.372380 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:54.372381 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:54.372388 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:54.373758 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:54.374214 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:54.374216 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:54.374216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:54.374315 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:54.374628 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:54.374630 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:54.374630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:54.374756 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:54.374835 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:54.375299 (+ 464us) server_negotiation.cc:299] Negotiation successful
0806 15:00:54.375398 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 15:00:54.375699 30961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:54.366742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.65:40099)
0806 15:00:54.366911 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:54.366954 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:54.367444 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:54.367656 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:54.367659 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:54.368253 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:00:54.368441 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:54.368447 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.369163 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.369164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:54.369937 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:54.369939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.370346 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.370347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:54.370468 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:54.370911 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:00:54.370920 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:00:54.372292 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:00:54.373839 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:54.373847 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:54.373849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:54.374112 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:54.374390 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:54.374391 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:54.374391 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:54.374566 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:54.375026 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:00:54.375032 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:00:54.375160 (+ 128us) client_negotiation.cc:769] Sending connection context
0806 15:00:54.375274 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 15:00:54.375432 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
I20260806 15:00:54.376212 30958 heartbeater.cc:347] Connected to a master server at 127.11.8.126:38995
I20260806 15:00:54.376322 30958 heartbeater.cc:464] Registering TS with master...
I20260806 15:00:54.376511 30958 heartbeater.cc:511] Master 127.11.8.126:38995 requested a full tablet report, sending...
I20260806 15:00:54.377094 30685 ts_manager.cc:194] Registered new tserver with Master: 39babaadc45f4e808c7cbb9b96805b7c (127.11.8.65:34479)
I20260806 15:00:54.377700 30685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.65@KRBTEST.COM'} at 127.11.8.65:40099
WARNING: no policy specified for kudu/127.11.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
I20260806 15:00:54.401777 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:0
--local_ip_for_outbound_sockets=127.11.8.66
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:38995
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:37615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:00:54.484179 30964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:00:54.484385 30964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:00:54.484408 30964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:00:54.485497 30964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:00:54.485534 30964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:00:54.485560 30964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 15:00:54.487006 30964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:37615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:38995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:00:54.487186 30964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:00:54.487355 30964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:00:54.492465 30978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:54.492504 30972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:54.492504 30971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:54.492903 30964 server_base.cc:1034] running on GCE node
I20260806 15:00:54.493113 30964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:00:54.493361 30964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:00:54.494470 30964 hybrid_clock.cc:648] HybridClock initialized: now 1786028454494463 us; error 45 us; skew 500 ppm
Aug 06 15:00:54 dist-test-slave-68xk krb5kdc[30611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028454, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:00:54.500841 30964 init.cc:377] Logged in from keytab as kudu/127.11.8.66@KRBTEST.COM (short username kudu)
I20260806 15:00:54.501314 30964 webserver.cc:460] Webserver started at http://127.11.8.66:43167/ using document root <none> and password file <none>
I20260806 15:00:54.501503 30964 fs_manager.cc:359] Metadata directory not provided
I20260806 15:00:54.501541 30964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:00:54.501637 30964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:00:54.502750 30964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/data/instance:
uuid: "4e9bb16103274469a57a60f4b12005fc"
format_stamp: "Formatted at 2026-08-06 15:00:54 on dist-test-slave-68xk"
server_key: "857015cc99ebfd6c1b16a1c5970bb958"
server_key_iv: "8ad05443322ec64461095a9ce287332a"
server_key_version: "encryptionkey@0"
I20260806 15:00:54.503053 30964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance:
uuid: "4e9bb16103274469a57a60f4b12005fc"
format_stamp: "Formatted at 2026-08-06 15:00:54 on dist-test-slave-68xk"
server_key: "857015cc99ebfd6c1b16a1c5970bb958"
server_key_iv: "8ad05443322ec64461095a9ce287332a"
server_key_version: "encryptionkey@0"
I20260806 15:00:54.504485 30964 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:00:54.505270 30990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:54.505491 30964 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:00:54.505585 30964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "4e9bb16103274469a57a60f4b12005fc"
format_stamp: "Formatted at 2026-08-06 15:00:54 on dist-test-slave-68xk"
server_key: "857015cc99ebfd6c1b16a1c5970bb958"
server_key_iv: "8ad05443322ec64461095a9ce287332a"
server_key_version: "encryptionkey@0"
I20260806 15:00:54.505645 30964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:00:54.554924 30964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:00:54.557763 30964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:00:54.557898 30964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:00:54.558121 30964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:00:54.558539 30964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:00:54.558588 30964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:54.558614 30964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:00:54.558629 30964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:54.565325 30964 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:44099
I20260806 15:00:54.565392 31152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:44099 every 8 connection(s)
I20260806 15:00:54.565743 30964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
I20260806 15:00:54.567361 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 30964
I20260806 15:00:54.567507 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance
I20260806 15:00:54.567728 11297 external_mini_cluster.cc:1428] Setting key af5a3fe6b3c1d746313c8befbd219372
Aug 06 15:00:54 dist-test-slave-68xk krb5kdc[30611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028454, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:00:54.576226 30743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:54.566531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:34923 (local address 127.11.8.126:38995)
0806 15:00:54.566610 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:54.566612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:54.566840 (+ 228us) server_negotiation.cc:407] Connection header received
0806 15:00:54.567424 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:54.567426 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:54.567439 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:54.567513 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:54.568783 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.569418 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.570589 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.570821 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.570913 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:54.573269 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:54.573271 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:54.573272 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:54.573281 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:54.574843 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:54.575271 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:54.575272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:54.575273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:54.575333 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:54.575585 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:54.575586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:54.575586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:54.575702 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:54.575777 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:54.576043 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 15:00:54.576131 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 15:00:54.576436 31156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:54.566518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.66:34923)
0806 15:00:54.566714 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:54.566755 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:54.567309 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:54.567592 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:54.567597 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:54.568460 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:00:54.568685 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:54.568691 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.569530 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.569532 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:54.570493 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:54.570495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.570924 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.570926 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:54.571071 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:54.571660 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:00:54.571669 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:00:54.573157 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:00:54.574936 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:54.574943 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:54.574945 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:54.575171 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:54.575405 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:54.575406 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:54.575406 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:54.575514 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:54.575767 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:00:54.575774 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:00:54.575914 (+ 140us) client_negotiation.cc:769] Sending connection context
0806 15:00:54.576022 (+ 108us) client_negotiation.cc:240] Negotiation successful
0806 15:00:54.576180 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":101,"threads_started":1}
WARNING: no policy specified for kudu/127.11.8.67@KRBTEST.COM; defaulting to no policy
I20260806 15:00:54.576903 31153 heartbeater.cc:347] Connected to a master server at 127.11.8.126:38995
I20260806 15:00:54.576993 31153 heartbeater.cc:464] Registering TS with master...
I20260806 15:00:54.577175 31153 heartbeater.cc:511] Master 127.11.8.126:38995 requested a full tablet report, sending...
I20260806 15:00:54.577613 30685 ts_manager.cc:194] Registered new tserver with Master: 4e9bb16103274469a57a60f4b12005fc (127.11.8.66:44099)
I20260806 15:00:54.578017 30685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.66@KRBTEST.COM'} at 127.11.8.66:34923
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
I20260806 15:00:54.595515 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:0
--local_ip_for_outbound_sockets=127.11.8.67
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:38995
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:37615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:00:54.693852 31160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:00:54.694088 31160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:00:54.694114 31160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:00:54.695327 31160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:00:54.695374 31160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:00:54.695400 31160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 15:00:54.696792 31160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:37615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:38995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:00:54.696974 31160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:00:54.697156 31160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:00:54.702116 31166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:54.702201 31160 server_base.cc:1034] running on GCE node
W20260806 15:00:54.702142 31168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:54.702117 31170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:54.702567 31160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:00:54.702817 31160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:00:54.703963 31160 hybrid_clock.cc:648] HybridClock initialized: now 1786028454703950 us; error 33 us; skew 500 ppm
Aug 06 15:00:54 dist-test-slave-68xk krb5kdc[30611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028454, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:00:54.706499 31160 init.cc:377] Logged in from keytab as kudu/127.11.8.67@KRBTEST.COM (short username kudu)
I20260806 15:00:54.706957 31160 webserver.cc:460] Webserver started at http://127.11.8.67:40093/ using document root <none> and password file <none>
I20260806 15:00:54.707201 31160 fs_manager.cc:359] Metadata directory not provided
I20260806 15:00:54.707269 31160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:00:54.707424 31160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:00:54.708443 31160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/data/instance:
uuid: "fb63490d5ac94a598651beff7b742c19"
format_stamp: "Formatted at 2026-08-06 15:00:54 on dist-test-slave-68xk"
server_key: "20c41eeea84e4117abc8676dc2739c08"
server_key_iv: "003429adc2b2bd71ab645bf8539b877b"
server_key_version: "encryptionkey@0"
I20260806 15:00:54.708860 31160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance:
uuid: "fb63490d5ac94a598651beff7b742c19"
format_stamp: "Formatted at 2026-08-06 15:00:54 on dist-test-slave-68xk"
server_key: "20c41eeea84e4117abc8676dc2739c08"
server_key_iv: "003429adc2b2bd71ab645bf8539b877b"
server_key_version: "encryptionkey@0"
I20260806 15:00:54.710230 31160 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:00:54.711033 31177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:54.711201 31160 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:54.711287 31160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "fb63490d5ac94a598651beff7b742c19"
format_stamp: "Formatted at 2026-08-06 15:00:54 on dist-test-slave-68xk"
server_key: "20c41eeea84e4117abc8676dc2739c08"
server_key_iv: "003429adc2b2bd71ab645bf8539b877b"
server_key_version: "encryptionkey@0"
I20260806 15:00:54.711369 31160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:00:54.723681 31160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:00:54.725996 31160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:00:54.726172 31160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:00:54.726452 31160 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:00:54.726835 31160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:00:54.726894 31160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:54.726936 31160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:00:54.726964 31160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:54.733330 31160 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:38519
I20260806 15:00:54.733383 31346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:38519 every 8 connection(s)
I20260806 15:00:54.733742 31160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
Aug 06 15:00:54 dist-test-slave-68xk krb5kdc[30611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028454, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:00:54.741035 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 31160
I20260806 15:00:54.741163 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance
I20260806 15:00:54.741382 11297 external_mini_cluster.cc:1428] Setting key 0aee34c482646b3d81e24d47e859b622
I20260806 15:00:54.743376 30743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:54.734575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:58477 (local address 127.11.8.126:38995)
0806 15:00:54.734645 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:54.734648 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:54.734867 (+ 219us) server_negotiation.cc:407] Connection header received
0806 15:00:54.735373 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:54.735374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:54.735385 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:54.735438 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:54.736412 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.736911 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.737889 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.738086 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.738162 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:54.740365 (+ 2203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:54.740367 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:54.740367 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:54.740374 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:54.741995 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:54.742525 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:54.742526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:54.742526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:54.742584 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:54.742853 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:54.742854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:54.742854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:54.742959 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:54.743012 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:54.743252 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 15:00:54.743313 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 15:00:54.743541 31350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:54.734573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.67:58477)
0806 15:00:54.734743 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:54.734779 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:54.735288 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:54.735520 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:54.735524 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:54.736123 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:00:54.736320 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:54.736325 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.736991 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.736992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:54.737785 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:54.737787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.738178 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.738179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:54.738285 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:54.738865 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:00:54.738874 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:00:54.740279 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:00:54.742071 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:54.742079 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:54.742081 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:54.742393 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:54.742630 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:54.742630 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:54.742631 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:54.742784 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:54.743018 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:00:54.743024 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:00:54.743139 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 15:00:54.743226 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 15:00:54.743347 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":94,"threads_started":1}
I20260806 15:00:54.744019 31347 heartbeater.cc:347] Connected to a master server at 127.11.8.126:38995
I20260806 15:00:54.744100 31347 heartbeater.cc:464] Registering TS with master...
I20260806 15:00:54.744262 31347 heartbeater.cc:511] Master 127.11.8.126:38995 requested a full tablet report, sending...
I20260806 15:00:54.744616 30685 ts_manager.cc:194] Registered new tserver with Master: fb63490d5ac94a598651beff7b742c19 (127.11.8.67:38519)
I20260806 15:00:54.744949 30685 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.67@KRBTEST.COM'} at 127.11.8.67:58477
I20260806 15:00:54.753042 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:00:54.760561 30743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:54.754398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47710 (local address 127.11.8.126:38995)
0806 15:00:54.754494 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:54.754496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:54.754645 (+ 149us) server_negotiation.cc:407] Connection header received
0806 15:00:54.754732 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:54.754732 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:54.754743 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:54.754801 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:54.755403 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.755943 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.756748 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.756981 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.757036 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:54.758188 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:54.758190 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:54.758190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:54.758198 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:54.759396 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:54.759742 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:54.759743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:54.759743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:54.759805 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:54.760002 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:54.760003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:54.760003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:54.760156 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:54.760258 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:54.760382 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 15:00:54.760460 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 15:00:54.761904 30685 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47710:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:00:54.763526 30685 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:00:54.770315 31364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:54.766011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:44099 (local address 127.0.0.1:55732)
0806 15:00:54.766310 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:54.766370 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:54.766469 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:54.766844 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:54.766848 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:54.766851 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:00:54.766938 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:54.766940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.767851 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.767852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:54.768868 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:54.768871 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.769983 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.769985 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:54.770073 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:54.770088 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 15:00:54.770167 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 15:00:54.770222 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":59,"threads_started":1}
I20260806 15:00:54.770335 31365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:54.766110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55732 (local address 127.11.8.66:44099)
0806 15:00:54.766472 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:54.766479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:54.766492 (+ 13us) server_negotiation.cc:407] Connection header received
0806 15:00:54.766638 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:54.766641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:54.766661 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:54.766745 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:00:54.767012 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.767747 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.768990 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.769845 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.769982 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:54.770071 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:54.770183 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 15:00:54.770239 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":86,"threads_started":1}
I20260806 15:00:54.770599 31367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:54.766062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:38519 (local address 127.0.0.1:47550)
0806 15:00:54.766589 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:54.766600 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:54.766668 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:54.767004 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:54.767005 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:54.767008 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:00:54.767068 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:54.767069 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.768525 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.768526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:54.769400 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:54.769402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.770166 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.770166 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:54.770222 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:54.770226 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:00:54.770274 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 15:00:54.770307 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"mutex_wait_us":39,"thread_start_us":104,"threads_started":1}
I20260806 15:00:54.770977 31369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:54.766102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47550 (local address 127.11.8.67:38519)
0806 15:00:54.766545 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:54.766551 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:54.766784 (+ 233us) server_negotiation.cc:407] Connection header received
0806 15:00:54.766835 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:54.766837 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:54.766855 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:54.766935 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:00:54.767222 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.768405 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.769487 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.770010 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.770076 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:54.770131 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:54.770360 (+ 229us) server_negotiation.cc:299] Negotiation successful
0806 15:00:54.770887 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":264,"threads_started":1}
I20260806 15:00:54.771198 31053 tablet_service.cc:1467] Processing CreateTablet for tablet 15e13586d43f42f3ab285bfec285c094 (DEFAULT_TABLE table=test-table [id=ef7f2bd6232f4257a5d10e3956822bf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:00:54.771647 31053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15e13586d43f42f3ab285bfec285c094. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:00:54.771703 31248 tablet_service.cc:1467] Processing CreateTablet for tablet 15e13586d43f42f3ab285bfec285c094 (DEFAULT_TABLE table=test-table [id=ef7f2bd6232f4257a5d10e3956822bf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:00:54.771848 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:54.766220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:34479 (local address 127.0.0.1:52578)
0806 15:00:54.766621 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:54.766630 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:54.766690 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:54.766995 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:54.766996 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:54.766999 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:00:54.767055 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:54.767057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.768556 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.768557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:54.769883 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:54.769887 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.771605 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.771606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:54.771666 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:54.771671 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 15:00:54.771734 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 15:00:54.771774 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":76,"threads_started":1}
I20260806 15:00:54.772081 31248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15e13586d43f42f3ab285bfec285c094. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:00:54.772099 31370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:54.766402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52578 (local address 127.11.8.65:34479)
0806 15:00:54.766579 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:54.766582 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:54.766756 (+ 174us) server_negotiation.cc:407] Connection header received
0806 15:00:54.766804 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:54.766807 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:54.766824 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:54.766897 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:00:54.767153 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.768405 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.770013 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.771525 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.771604 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:54.771659 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:54.771813 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 15:00:54.771860 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":106,"threads_started":1}
I20260806 15:00:54.772647 30859 tablet_service.cc:1467] Processing CreateTablet for tablet 15e13586d43f42f3ab285bfec285c094 (DEFAULT_TABLE table=test-table [id=ef7f2bd6232f4257a5d10e3956822bf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:00:54.772941 30859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15e13586d43f42f3ab285bfec285c094. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:00:54.775527 31371 tablet_bootstrap.cc:492] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c: Bootstrap starting.
I20260806 15:00:54.775835 31374 tablet_bootstrap.cc:492] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc: Bootstrap starting.
I20260806 15:00:54.776222 31371 tablet_bootstrap.cc:654] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c: Neither blocks nor log segments found. Creating new log.
I20260806 15:00:54.776494 31374 tablet_bootstrap.cc:654] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc: Neither blocks nor log segments found. Creating new log.
I20260806 15:00:54.776746 31371 log.cc:824] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c: Log is configured to *not* fsync() on all Append() calls
I20260806 15:00:54.777070 31372 tablet_bootstrap.cc:492] T 15e13586d43f42f3ab285bfec285c094 P fb63490d5ac94a598651beff7b742c19: Bootstrap starting.
I20260806 15:00:54.777190 31374 log.cc:824] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc: Log is configured to *not* fsync() on all Append() calls
I20260806 15:00:54.777663 31371 tablet_bootstrap.cc:492] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c: No bootstrap required, opened a new log
I20260806 15:00:54.777787 31371 ts_tablet_manager.cc:1397] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:00:54.777824 31372 tablet_bootstrap.cc:654] T 15e13586d43f42f3ab285bfec285c094 P fb63490d5ac94a598651beff7b742c19: Neither blocks nor log segments found. Creating new log.
I20260806 15:00:54.778146 31374 tablet_bootstrap.cc:492] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc: No bootstrap required, opened a new log
I20260806 15:00:54.778237 31374 ts_tablet_manager.cc:1397] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:00:54.778399 31372 log.cc:824] T 15e13586d43f42f3ab285bfec285c094 P fb63490d5ac94a598651beff7b742c19: Log is configured to *not* fsync() on all Append() calls
I20260806 15:00:54.779210 31372 tablet_bootstrap.cc:492] T 15e13586d43f42f3ab285bfec285c094 P fb63490d5ac94a598651beff7b742c19: No bootstrap required, opened a new log
I20260806 15:00:54.779284 31372 ts_tablet_manager.cc:1397] T 15e13586d43f42f3ab285bfec285c094 P fb63490d5ac94a598651beff7b742c19: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 15:00:54.779953 31371 raft_consensus.cc:348] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } }
I20260806 15:00:54.780125 31371 raft_consensus.cc:374] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:00:54.780176 31371 raft_consensus.cc:729] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39babaadc45f4e808c7cbb9b96805b7c, State: Initialized, Role: FOLLOWER
I20260806 15:00:54.780251 31374 raft_consensus.cc:348] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } }
I20260806 15:00:54.780337 31371 consensus_queue.cc:260] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } }
I20260806 15:00:54.780431 31374 raft_consensus.cc:374] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:00:54.780481 31374 raft_consensus.cc:729] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e9bb16103274469a57a60f4b12005fc, State: Initialized, Role: FOLLOWER
I20260806 15:00:54.780606 31374 consensus_queue.cc:260] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } }
I20260806 15:00:54.780627 31371 ts_tablet_manager.cc:1428] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 15:00:54.780802 30958 heartbeater.cc:503] Master 127.11.8.126:38995 was elected leader, sending a full tablet report...
I20260806 15:00:54.780920 31374 ts_tablet_manager.cc:1428] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 15:00:54.780936 31153 heartbeater.cc:503] Master 127.11.8.126:38995 was elected leader, sending a full tablet report...
I20260806 15:00:54.781500 31372 raft_consensus.cc:348] T 15e13586d43f42f3ab285bfec285c094 P fb63490d5ac94a598651beff7b742c19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } }
I20260806 15:00:54.781639 31372 raft_consensus.cc:374] T 15e13586d43f42f3ab285bfec285c094 P fb63490d5ac94a598651beff7b742c19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:00:54.781679 31372 raft_consensus.cc:729] T 15e13586d43f42f3ab285bfec285c094 P fb63490d5ac94a598651beff7b742c19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb63490d5ac94a598651beff7b742c19, State: Initialized, Role: FOLLOWER
I20260806 15:00:54.781814 31372 consensus_queue.cc:260] T 15e13586d43f42f3ab285bfec285c094 P fb63490d5ac94a598651beff7b742c19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } }
I20260806 15:00:54.782120 31372 ts_tablet_manager.cc:1428] T 15e13586d43f42f3ab285bfec285c094 P fb63490d5ac94a598651beff7b742c19: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 15:00:54.782240 31347 heartbeater.cc:503] Master 127.11.8.126:38995 was elected leader, sending a full tablet report...
W20260806 15:00:54.816335 31155 tablet.cc:2369] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 15:00:54.866868 30959 tablet.cc:2369] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:00:54.901206 31381 raft_consensus.cc:482] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:00:54.901340 31381 raft_consensus.cc:504] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } }
I20260806 15:00:54.901835 31381 leader_election.cc:288] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb63490d5ac94a598651beff7b742c19 (127.11.8.67:38519), 39babaadc45f4e808c7cbb9b96805b7c (127.11.8.65:34479)
I20260806 15:00:54.905048 31370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:54.902151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:55867 (local address 127.11.8.65:34479)
0806 15:00:54.902249 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:54.902252 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:54.902551 (+ 299us) server_negotiation.cc:407] Connection header received
0806 15:00:54.902627 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:54.902628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:54.902641 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:54.902713 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:00:54.902879 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.903358 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.904312 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.904697 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.904781 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:54.904792 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:54.904929 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 15:00:54.904971 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 15:00:54.905058 31383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:54.902071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:34479 (local address 127.11.8.66:55867)
0806 15:00:54.902465 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:54.902479 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:54.902558 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:54.902732 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:54.902733 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:54.902735 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:00:54.902808 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:54.902810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.903428 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.903429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:54.904219 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:54.904221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.904783 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.904784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:54.904854 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:54.904857 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:00:54.904922 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 15:00:54.904962 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":315,"threads_started":1}
I20260806 15:00:54.905158 31156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:54.902051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:38519 (local address 127.11.8.66:53751)
0806 15:00:54.902104 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:54.902119 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:54.902177 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:54.902399 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:54.902400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:54.902402 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:00:54.902504 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:54.902506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.903361 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.903363 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:54.904216 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:54.904219 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.904841 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.904842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:54.904929 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:54.904932 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:00:54.905004 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 15:00:54.905054 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 15:00:54.905148 31369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:54.902201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:53751 (local address 127.11.8.67:38519)
0806 15:00:54.902255 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:54.902258 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:54.902268 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:00:54.902301 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:54.902302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:54.902314 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:54.902392 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:00:54.902586 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.903225 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.904325 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.904753 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.904841 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:54.904853 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:54.905010 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 15:00:54.905061 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 15:00:54.905491 30893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15e13586d43f42f3ab285bfec285c094" candidate_uuid: "4e9bb16103274469a57a60f4b12005fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39babaadc45f4e808c7cbb9b96805b7c" is_pre_election: true
I20260806 15:00:54.905654 30893 raft_consensus.cc:2454] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e9bb16103274469a57a60f4b12005fc in term 0.
I20260806 15:00:54.905725 31278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15e13586d43f42f3ab285bfec285c094" candidate_uuid: "4e9bb16103274469a57a60f4b12005fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb63490d5ac94a598651beff7b742c19" is_pre_election: true
I20260806 15:00:54.905912 31278 raft_consensus.cc:2454] T 15e13586d43f42f3ab285bfec285c094 P fb63490d5ac94a598651beff7b742c19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e9bb16103274469a57a60f4b12005fc in term 0.
I20260806 15:00:54.905965 30994 leader_election.cc:302] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39babaadc45f4e808c7cbb9b96805b7c, 4e9bb16103274469a57a60f4b12005fc; no voters:
I20260806 15:00:54.906118 31381 raft_consensus.cc:2781] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:00:54.906179 31381 raft_consensus.cc:482] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:00:54.906196 31381 raft_consensus.cc:3037] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:00:54.906975 31381 raft_consensus.cc:504] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } }
I20260806 15:00:54.907119 31381 leader_election.cc:288] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc [CANDIDATE]: Term 1 election: Requested vote from peers fb63490d5ac94a598651beff7b742c19 (127.11.8.67:38519), 39babaadc45f4e808c7cbb9b96805b7c (127.11.8.65:34479)
I20260806 15:00:54.907305 31278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15e13586d43f42f3ab285bfec285c094" candidate_uuid: "4e9bb16103274469a57a60f4b12005fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb63490d5ac94a598651beff7b742c19"
I20260806 15:00:54.907311 30893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15e13586d43f42f3ab285bfec285c094" candidate_uuid: "4e9bb16103274469a57a60f4b12005fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39babaadc45f4e808c7cbb9b96805b7c"
I20260806 15:00:54.907392 31278 raft_consensus.cc:3037] T 15e13586d43f42f3ab285bfec285c094 P fb63490d5ac94a598651beff7b742c19 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:00:54.907429 30893 raft_consensus.cc:3037] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:00:54.908139 31278 raft_consensus.cc:2454] T 15e13586d43f42f3ab285bfec285c094 P fb63490d5ac94a598651beff7b742c19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e9bb16103274469a57a60f4b12005fc in term 1.
I20260806 15:00:54.908360 30995 leader_election.cc:302] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e9bb16103274469a57a60f4b12005fc, fb63490d5ac94a598651beff7b742c19; no voters:
I20260806 15:00:54.908456 30893 raft_consensus.cc:2454] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e9bb16103274469a57a60f4b12005fc in term 1.
I20260806 15:00:54.908475 31381 raft_consensus.cc:2781] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:00:54.908677 31381 raft_consensus.cc:686] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc [term 1 LEADER]: Becoming Leader. State: Replica: 4e9bb16103274469a57a60f4b12005fc, State: Running, Role: LEADER
I20260806 15:00:54.908813 31381 consensus_queue.cc:237] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } }
I20260806 15:00:54.909756 30683 catalog_manager.cc:5495] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e9bb16103274469a57a60f4b12005fc (127.11.8.66). New cstate: current_term: 1 leader_uuid: "4e9bb16103274469a57a60f4b12005fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } health_report { overall_health: UNKNOWN } } }
I20260806 15:00:54.914592 31365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:54.912324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55742 (local address 127.11.8.66:44099)
0806 15:00:54.912370 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:54.912372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:54.912382 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:00:54.912423 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:54.912423 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:54.912440 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:54.912507 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:00:54.912628 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.913094 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.913786 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:54.914019 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:54.914075 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:54.914232 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:00:54.914324 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:00:54.914387 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:54.914465 (+ 78us) server_negotiation.cc:299] Negotiation successful
0806 15:00:54.914483 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:00:54.916287 30893 raft_consensus.cc:1261] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c [term 1 FOLLOWER]: Refusing update from remote peer 4e9bb16103274469a57a60f4b12005fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:00:54.916306 31278 raft_consensus.cc:1261] T 15e13586d43f42f3ab285bfec285c094 P fb63490d5ac94a598651beff7b742c19 [term 1 FOLLOWER]: Refusing update from remote peer 4e9bb16103274469a57a60f4b12005fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:00:54.916608 31384 consensus_queue.cc:1035] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:00:54.916754 31381 consensus_queue.cc:1035] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:00:54.919386 31392 mvcc.cc:205] Tried to move back new op lower bound from 7315572551334707200 to 7315572551307276288. Current Snapshot: MvccSnapshot[applied={T|T < 7315572551334707200}]
I20260806 15:00:54.919708 31389 mvcc.cc:205] Tried to move back new op lower bound from 7315572551334707200 to 7315572551307276288. Current Snapshot: MvccSnapshot[applied={T|T < 7315572551334707200}]
I20260806 15:00:54.921470 30685 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:00:54.921567 30685 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:00:54.922281 30685 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30: Sending DeleteTablet for 3 replicas of tablet 15e13586d43f42f3ab285bfec285c094
I20260806 15:00:54.922699 31053 tablet_service.cc:1514] Processing DeleteTablet for tablet 15e13586d43f42f3ab285bfec285c094 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:00:54 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:55732
I20260806 15:00:54.922767 31248 tablet_service.cc:1514] Processing DeleteTablet for tablet 15e13586d43f42f3ab285bfec285c094 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:00:54 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:47550
I20260806 15:00:54.922953 31407 tablet_replica.cc:330] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc: stopping tablet replica
I20260806 15:00:54.922968 30859 tablet_service.cc:1514] Processing DeleteTablet for tablet 15e13586d43f42f3ab285bfec285c094 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:00:54 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:52578
I20260806 15:00:54.922986 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 30770
I20260806 15:00:54.923053 31407 raft_consensus.cc:2229] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:00:54.923138 31407 raft_consensus.cc:2258] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:00:54.928194 31407 ts_tablet_manager.cc:1905] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:00:54.930446 31407 ts_tablet_manager.cc:1918] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:00:54.930518 31407 log.cc:1196] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/wal/wals/15e13586d43f42f3ab285bfec285c094
I20260806 15:00:54.930801 31407 ts_tablet_manager.cc:1939] T 15e13586d43f42f3ab285bfec285c094 P 4e9bb16103274469a57a60f4b12005fc: Deleting consensus metadata
I20260806 15:00:54.932094 30660 catalog_manager.cc:4841] TS 4e9bb16103274469a57a60f4b12005fc (127.11.8.66:44099): tablet 15e13586d43f42f3ab285bfec285c094 (table test-table [id=ef7f2bd6232f4257a5d10e3956822bf7]) successfully deleted
I20260806 15:00:54.932916 31408 tablet_replica.cc:330] T 15e13586d43f42f3ab285bfec285c094 P fb63490d5ac94a598651beff7b742c19: stopping tablet replica
I20260806 15:00:54.933046 31408 raft_consensus.cc:2229] T 15e13586d43f42f3ab285bfec285c094 P fb63490d5ac94a598651beff7b742c19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:00:54.933174 31408 raft_consensus.cc:2258] T 15e13586d43f42f3ab285bfec285c094 P fb63490d5ac94a598651beff7b742c19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:00:54.933593 31408 ts_tablet_manager.cc:1905] T 15e13586d43f42f3ab285bfec285c094 P fb63490d5ac94a598651beff7b742c19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:00:54.934891 31408 ts_tablet_manager.cc:1918] T 15e13586d43f42f3ab285bfec285c094 P fb63490d5ac94a598651beff7b742c19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:00:54.934983 31408 log.cc:1196] T 15e13586d43f42f3ab285bfec285c094 P fb63490d5ac94a598651beff7b742c19: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/wal/wals/15e13586d43f42f3ab285bfec285c094
I20260806 15:00:54.935258 31408 ts_tablet_manager.cc:1939] T 15e13586d43f42f3ab285bfec285c094 P fb63490d5ac94a598651beff7b742c19: Deleting consensus metadata
I20260806 15:00:54.935653 30661 catalog_manager.cc:4841] TS fb63490d5ac94a598651beff7b742c19 (127.11.8.67:38519): tablet 15e13586d43f42f3ab285bfec285c094 (table test-table [id=ef7f2bd6232f4257a5d10e3956822bf7]) successfully deleted
W20260806 15:00:54.938922 30660 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.65:34479 (error 108)
I20260806 15:00:54.939131 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 30964
I20260806 15:00:54.944422 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:54.944286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:34479 (local address 127.0.0.1:52594)
0806 15:00:54.944340 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:54.944381 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.65:34479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 15:00:54.944541 30660 catalog_manager.cc:4584] TS 39babaadc45f4e808c7cbb9b96805b7c (127.11.8.65:34479): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 15e13586d43f42f3ab285bfec285c094: Network error: Client connection negotiation failed: client connection to 127.11.8.65:34479: connect: Connection refused (error 111)
I20260806 15:00:54.955260 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 31160
I20260806 15:00:54.964869 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 30636
I20260806 15:00:54.976969 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:38995
--webserver_interface=127.11.8.126
--webserver_port=32903
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:37615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:38995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:00:55.071558 31410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:00:55.071808 31410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:00:55.071849 31410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:00:55.073058 31410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:00:55.073127 31410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:00:55.073153 31410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:00:55.073180 31410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:00:55.073210 31410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:00:55.074710 31410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:37615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:38995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:38995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=32903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:00:55.074990 31410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:00:55.075220 31410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:00:55.084597 31417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:55.084741 31410 server_base.cc:1034] running on GCE node
W20260806 15:00:55.084641 31418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:55.084604 31420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:55.085194 31410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:00:55.085520 31410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:00:55.086695 31410 hybrid_clock.cc:648] HybridClock initialized: now 1786028455086680 us; error 40 us; skew 500 ppm
Aug 06 15:00:55 dist-test-slave-68xk krb5kdc[30611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028455, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:00:55.089308 31410 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 15:00:55.089792 31410 webserver.cc:460] Webserver started at http://127.11.8.126:32903/ using document root <none> and password file <none>
I20260806 15:00:55.090042 31410 fs_manager.cc:359] Metadata directory not provided
I20260806 15:00:55.090129 31410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:00:55.091584 31410 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:00:55.092559 31429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:55.092777 31410 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:00:55.092904 31410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "2f76c3cc03194bdcb237406279446e30"
format_stamp: "Formatted at 2026-08-06 15:00:54 on dist-test-slave-68xk"
server_key: "68b434bd754da26e7bed7757697f78fa"
server_key_iv: "e1ece243453028139ea624c9e6a98846"
server_key_version: "encryptionkey@0"
I20260806 15:00:55.093386 31410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:00:55.103379 31410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:00:55.105506 31410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:00:55.105713 31410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:00:55.110088 31410 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:38995
I20260806 15:00:55.110122 31506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:38995 every 8 connection(s)
I20260806 15:00:55.110548 31410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:00:55.112197 31507 sys_catalog.cc:263] Verifying existing consensus state
I20260806 15:00:55.112898 31507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30: Bootstrap starting.
I20260806 15:00:55.113108 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 31410
I20260806 15:00:55.113411 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:34479
--local_ip_for_outbound_sockets=127.11.8.65
--tserver_master_addrs=127.11.8.126:38995
--webserver_port=32789
--webserver_interface=127.11.8.65
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:37615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:00:55.121941 31507 log.cc:824] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30: Log is configured to *not* fsync() on all Append() calls
I20260806 15:00:55.123900 31507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 15:00:55.124203 31507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30: Bootstrap complete.
I20260806 15:00:55.126632 31507 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f76c3cc03194bdcb237406279446e30" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38995 } }
I20260806 15:00:55.126900 31507 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f76c3cc03194bdcb237406279446e30, State: Initialized, Role: FOLLOWER
I20260806 15:00:55.127101 31507 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f76c3cc03194bdcb237406279446e30" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38995 } }
I20260806 15:00:55.127194 31507 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:00:55.127254 31507 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:00:55.127305 31507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [term 1 FOLLOWER]: Advancing to term 2
I20260806 15:00:55.128304 31507 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f76c3cc03194bdcb237406279446e30" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38995 } }
I20260806 15:00:55.128420 31507 leader_election.cc:302] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f76c3cc03194bdcb237406279446e30; no voters:
I20260806 15:00:55.128605 31507 leader_election.cc:288] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 15:00:55.128676 31511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 15:00:55.128829 31511 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [term 2 LEADER]: Becoming Leader. State: Replica: 2f76c3cc03194bdcb237406279446e30, State: Running, Role: LEADER
I20260806 15:00:55.128894 31507 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:00:55.129033 31511 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f76c3cc03194bdcb237406279446e30" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38995 } }
I20260806 15:00:55.129421 31514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f76c3cc03194bdcb237406279446e30. Latest consensus state: current_term: 2 leader_uuid: "2f76c3cc03194bdcb237406279446e30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f76c3cc03194bdcb237406279446e30" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38995 } } }
I20260806 15:00:55.129498 31514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [sys.catalog]: This master's current role is: LEADER
I20260806 15:00:55.129710 31512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2f76c3cc03194bdcb237406279446e30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f76c3cc03194bdcb237406279446e30" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38995 } } }
I20260806 15:00:55.129814 31512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [sys.catalog]: This master's current role is: LEADER
I20260806 15:00:55.129966 31519 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:00:55.130810 31519 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:00:55.131080 31519 catalog_manager.cc:1259] Loaded cluster ID: f2ad979d149846c9a59bb665feff6281
I20260806 15:00:55.131215 31519 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:00:55.131479 31519 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 15:00:55.136238 31519 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 15:00:55.139329 31519 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:00:55.139549 31519 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30: Loaded TSK: 0
I20260806 15:00:55.139755 31519 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 15:00:55.207360 31509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:00:55.207561 31509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:00:55.207584 31509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:00:55.208758 31509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:00:55.208803 31509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:00:55.208830 31509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 15:00:55.210297 31509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:37615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:34479
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=32789
--tserver_master_addrs=127.11.8.126:38995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:00:55.210584 31509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:00:55.210826 31509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:00:55.215795 31542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:55.215785 31540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:55.215785 31538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:55.216195 31509 server_base.cc:1034] running on GCE node
I20260806 15:00:55.216391 31509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:00:55.216658 31509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:00:55.217825 31509 hybrid_clock.cc:648] HybridClock initialized: now 1786028455217814 us; error 28 us; skew 500 ppm
Aug 06 15:00:55 dist-test-slave-68xk krb5kdc[30611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028455, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:00:55.220485 31509 init.cc:377] Logged in from keytab as kudu/127.11.8.65@KRBTEST.COM (short username kudu)
I20260806 15:00:55.220896 31509 webserver.cc:460] Webserver started at http://127.11.8.65:32789/ using document root <none> and password file <none>
I20260806 15:00:55.221140 31509 fs_manager.cc:359] Metadata directory not provided
I20260806 15:00:55.221235 31509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:00:55.222648 31509 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:00:55.223376 31550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:55.223543 31509 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:00:55.223634 31509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "39babaadc45f4e808c7cbb9b96805b7c"
format_stamp: "Formatted at 2026-08-06 15:00:54 on dist-test-slave-68xk"
server_key: "a6b677c8b0f284677a2e52a57b42da2d"
server_key_iv: "3b66dac8d3c8158f34911f5d3118c833"
server_key_version: "encryptionkey@0"
I20260806 15:00:55.223973 31509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:00:55.236306 31509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:00:55.238505 31509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:00:55.238658 31509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:00:55.238883 31509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:00:55.239328 31563 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 15:00:55.241192 31509 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 15:00:55.241266 31509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 15:00:55.241317 31509 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 15:00:55.241952 31509 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 15:00:55.242007 31509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 15:00:55.242072 31563 tablet_bootstrap.cc:492] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c: Bootstrap starting.
I20260806 15:00:55.249547 31509 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:34479
I20260806 15:00:55.249951 31509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
I20260806 15:00:55.253237 31563 log.cc:824] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c: Log is configured to *not* fsync() on all Append() calls
I20260806 15:00:55.254216 31706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:34479 every 8 connection(s)
I20260806 15:00:55.255757 31563 tablet_bootstrap.cc:492] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 15:00:55.256050 31563 tablet_bootstrap.cc:492] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c: Bootstrap complete.
I20260806 15:00:55.257339 31563 ts_tablet_manager.cc:1397] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.009s
I20260806 15:00:55.258646 31563 raft_consensus.cc:348] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } }
I20260806 15:00:55.258956 31563 raft_consensus.cc:729] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39babaadc45f4e808c7cbb9b96805b7c, State: Initialized, Role: FOLLOWER
I20260806 15:00:55.259101 31563 consensus_queue.cc:260] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } }
I20260806 15:00:55.259402 31563 ts_tablet_manager.cc:1428] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
Aug 06 15:00:55 dist-test-slave-68xk krb5kdc[30611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028455, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:00:55.259820 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 31509
I20260806 15:00:55.260100 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:44099
--local_ip_for_outbound_sockets=127.11.8.66
--tserver_master_addrs=127.11.8.126:38995
--webserver_port=43167
--webserver_interface=127.11.8.66
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:37615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:00:55.264456 31709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:55.250720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.65:44177)
0806 15:00:55.253358 (+ 2638us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:55.253407 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:55.253986 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:55.254655 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:55.254660 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:55.254931 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:00:55.255122 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:55.255126 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.256399 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.256400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:55.257205 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:55.257207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.257859 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.257861 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:55.258002 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:55.258577 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:00:55.258586 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:00:55.260321 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:00:55.262684 (+ 2363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:55.262691 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:55.262694 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:55.262919 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:55.263186 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:55.263187 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:55.263187 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:55.263295 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:55.263761 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:00:55.263768 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:00:55.263925 (+ 157us) client_negotiation.cc:769] Sending connection context
0806 15:00:55.264012 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 15:00:55.264206 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2585,"thread_start_us":107,"threads_started":1}
I20260806 15:00:55.265327 31711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:55.250789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:44177 (local address 127.11.8.126:38995)
0806 15:00:55.253524 (+ 2735us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:55.253534 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:55.253579 (+ 45us) server_negotiation.cc:407] Connection header received
0806 15:00:55.254261 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:55.254269 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:55.254395 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:55.254579 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:55.255250 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.256293 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.257294 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.257748 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.257879 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:55.260443 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:55.260448 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:55.260448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:55.260462 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:55.262597 (+ 2135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:55.263016 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:55.263019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:55.263020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:55.263118 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:55.263371 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:55.263372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:55.263372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:55.263669 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:55.264807 (+ 1138us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:55.264894 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 15:00:55.265086 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2687,"thread_start_us":90,"threads_started":1}
I20260806 15:00:55.266101 31707 heartbeater.cc:347] Connected to a master server at 127.11.8.126:38995
I20260806 15:00:55.266206 31707 heartbeater.cc:464] Registering TS with master...
I20260806 15:00:55.266402 31707 heartbeater.cc:511] Master 127.11.8.126:38995 requested a full tablet report, sending...
I20260806 15:00:55.267268 31451 ts_manager.cc:194] Registered new tserver with Master: 39babaadc45f4e808c7cbb9b96805b7c (127.11.8.65:34479)
I20260806 15:00:55.268564 31451 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.65@KRBTEST.COM'} at 127.11.8.65:44177
I20260806 15:00:55.269630 31707 heartbeater.cc:503] Master 127.11.8.126:38995 was elected leader, sending a full tablet report...
Aug 06 15:00:55 dist-test-slave-68xk krb5kdc[30611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028455, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for kudu/127.11.8.65@KRBTEST.COM
I20260806 15:00:55.277470 31718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:55.268138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52598 (local address 127.11.8.65:34479)
0806 15:00:55.268495 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:55.268499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:55.268507 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:00:55.268536 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:55.268538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:55.268550 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:55.268608 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:55.269353 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.270646 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.271848 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.272261 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.272331 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:55.274775 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:55.274782 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:55.274782 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:55.274788 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:55.276312 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:55.276740 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:55.276742 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:55.276742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:55.276792 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:55.276975 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:55.276976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:55.276976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:55.277096 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:55.277156 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:55.277337 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 15:00:55.277410 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":70,"threads_started":1}
I20260806 15:00:55.277516 31717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:55.267922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:34479 (local address 127.0.0.1:52598)
0806 15:00:55.268294 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:55.268328 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:55.268388 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:55.268662 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:55.268666 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:55.269202 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:00:55.269268 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:55.269271 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.270726 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.270728 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:55.271732 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:55.271736 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.272350 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.272352 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:55.272443 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:55.272992 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:00:55.272998 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:00:55.274693 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:00:55.276389 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:55.276396 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:55.276398 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:55.276658 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:55.276850 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:55.276851 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:55.276851 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:55.276917 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:55.277160 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:00:55.277166 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:00:55.277218 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 15:00:55.277303 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 15:00:55.277416 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":76,"threads_started":1}
I20260806 15:00:55.277910 31616 tablet_service.cc:1514] Processing DeleteTablet for tablet 15e13586d43f42f3ab285bfec285c094 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:00:54 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:52598
I20260806 15:00:55.278527 31719 tablet_replica.cc:330] stopping tablet replica
I20260806 15:00:55.278649 31719 raft_consensus.cc:2229] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:00:55.278748 31719 raft_consensus.cc:2258] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:00:55.279027 31719 ts_tablet_manager.cc:1905] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:00:55.280999 31719 ts_tablet_manager.cc:1918] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:00:55.281078 31719 log.cc:1196] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/wal/wals/15e13586d43f42f3ab285bfec285c094
I20260806 15:00:55.281307 31719 ts_tablet_manager.cc:1939] T 15e13586d43f42f3ab285bfec285c094 P 39babaadc45f4e808c7cbb9b96805b7c: Deleting consensus metadata
I20260806 15:00:55.281780 31432 catalog_manager.cc:4841] TS 39babaadc45f4e808c7cbb9b96805b7c (127.11.8.65:34479): tablet 15e13586d43f42f3ab285bfec285c094 (table test-table [id=ef7f2bd6232f4257a5d10e3956822bf7]) successfully deleted
W20260806 15:00:55.348272 31715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:00:55.348515 31715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:00:55.348554 31715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:00:55.349700 31715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:00:55.349764 31715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:00:55.349807 31715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 15:00:55.351284 31715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:37615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:44099
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=43167
--tserver_master_addrs=127.11.8.126:38995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:00:55.351547 31715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:00:55.351783 31715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:00:55.356938 31727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:55.357007 31725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:55.356947 31730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:55.357141 31715 server_base.cc:1034] running on GCE node
I20260806 15:00:55.357317 31715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:00:55.357502 31715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:00:55.358705 31715 hybrid_clock.cc:648] HybridClock initialized: now 1786028455358690 us; error 29 us; skew 500 ppm
Aug 06 15:00:55 dist-test-slave-68xk krb5kdc[30611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028455, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:00:55.361374 31715 init.cc:377] Logged in from keytab as kudu/127.11.8.66@KRBTEST.COM (short username kudu)
I20260806 15:00:55.361866 31715 webserver.cc:460] Webserver started at http://127.11.8.66:43167/ using document root <none> and password file <none>
I20260806 15:00:55.362123 31715 fs_manager.cc:359] Metadata directory not provided
I20260806 15:00:55.362211 31715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:00:55.363595 31715 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:00:55.364197 31741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:55.364359 31715 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:00:55.364461 31715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "4e9bb16103274469a57a60f4b12005fc"
format_stamp: "Formatted at 2026-08-06 15:00:54 on dist-test-slave-68xk"
server_key: "857015cc99ebfd6c1b16a1c5970bb958"
server_key_iv: "8ad05443322ec64461095a9ce287332a"
server_key_version: "encryptionkey@0"
I20260806 15:00:55.364778 31715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:00:55.373026 31715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:00:55.374890 31715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:00:55.375041 31715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:00:55.375247 31715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:00:55.375545 31715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:00:55.375581 31715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:55.375602 31715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:00:55.375613 31715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:55.381634 31715 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:44099
I20260806 15:00:55.381680 31908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:44099 every 8 connection(s)
I20260806 15:00:55.381999 31715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
I20260806 15:00:55.386916 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 31715
I20260806 15:00:55.387228 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:38519
--local_ip_for_outbound_sockets=127.11.8.67
--tserver_master_addrs=127.11.8.126:38995
--webserver_port=40093
--webserver_interface=127.11.8.67
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:37615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 15:00:55 dist-test-slave-68xk krb5kdc[30611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028455, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:00:55.391664 31911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:55.382660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.66:51475)
0806 15:00:55.382854 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:55.382889 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:55.383348 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:55.383548 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:55.383551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:55.384103 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:00:55.384287 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:55.384291 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.384919 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.384920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:55.385778 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:55.385780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.386113 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.386115 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:55.386223 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:55.386683 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:00:55.386694 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:00:55.388304 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:00:55.390205 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:55.390212 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:55.390214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:55.390501 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:55.390763 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:55.390764 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:55.390764 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:55.390872 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:55.391180 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:00:55.391186 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:00:55.391300 (+ 114us) client_negotiation.cc:769] Sending connection context
0806 15:00:55.391387 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 15:00:55.391507 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":117,"threads_started":1}
I20260806 15:00:55.391669 31711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:55.382679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:51475 (local address 127.11.8.126:38995)
0806 15:00:55.382756 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:55.382759 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:55.382968 (+ 209us) server_negotiation.cc:407] Connection header received
0806 15:00:55.383427 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:55.383428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:55.383440 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:55.383491 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:55.384361 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.384842 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.385850 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.386035 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.386086 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:55.388404 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:55.388405 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:55.388406 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:55.388419 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:55.390113 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:55.390594 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:55.390596 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:55.390596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:55.390661 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:55.390946 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:55.390946 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:55.390947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:55.391109 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:55.391180 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:55.391427 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 15:00:55.391528 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 15:00:55.392379 31909 heartbeater.cc:347] Connected to a master server at 127.11.8.126:38995
I20260806 15:00:55.392469 31909 heartbeater.cc:464] Registering TS with master...
I20260806 15:00:55.392668 31909 heartbeater.cc:511] Master 127.11.8.126:38995 requested a full tablet report, sending...
I20260806 15:00:55.393059 31451 ts_manager.cc:194] Registered new tserver with Master: 4e9bb16103274469a57a60f4b12005fc (127.11.8.66:44099)
I20260806 15:00:55.393458 31451 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.66@KRBTEST.COM'} at 127.11.8.66:51475
W20260806 15:00:55.470578 31912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:00:55.470806 31912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:00:55.470844 31912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:00:55.471956 31912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:00:55.472023 31912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:00:55.472065 31912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 15:00:55.473496 31912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:37615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:38519
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=40093
--tserver_master_addrs=127.11.8.126:38995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:00:55.473770 31912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:00:55.474002 31912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:00:55.478929 31917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:55.479029 31912 server_base.cc:1034] running on GCE node
W20260806 15:00:55.478932 31918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:55.478929 31920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:55.479334 31912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:00:55.479589 31912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:00:55.480752 31912 hybrid_clock.cc:648] HybridClock initialized: now 1786028455480753 us; error 50 us; skew 500 ppm
Aug 06 15:00:55 dist-test-slave-68xk krb5kdc[30611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028455, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:00:55.483319 31912 init.cc:377] Logged in from keytab as kudu/127.11.8.67@KRBTEST.COM (short username kudu)
I20260806 15:00:55.483804 31912 webserver.cc:460] Webserver started at http://127.11.8.67:40093/ using document root <none> and password file <none>
I20260806 15:00:55.484047 31912 fs_manager.cc:359] Metadata directory not provided
I20260806 15:00:55.484128 31912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:00:55.485491 31912 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:00:55.486117 31931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:55.486295 31912 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:00:55.486397 31912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "fb63490d5ac94a598651beff7b742c19"
format_stamp: "Formatted at 2026-08-06 15:00:54 on dist-test-slave-68xk"
server_key: "20c41eeea84e4117abc8676dc2739c08"
server_key_iv: "003429adc2b2bd71ab645bf8539b877b"
server_key_version: "encryptionkey@0"
I20260806 15:00:55.486788 31912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:00:55.495536 31912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:00:55.497498 31912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:00:55.497699 31912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:00:55.497926 31912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:00:55.498235 31912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:00:55.498287 31912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:55.498327 31912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:00:55.498356 31912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:55.504523 31912 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:38519
I20260806 15:00:55.504572 32101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:38519 every 8 connection(s)
I20260806 15:00:55.504940 31912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
Aug 06 15:00:55 dist-test-slave-68xk krb5kdc[30611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028455, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:00:55.513808 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 31912
I20260806 15:00:55.514549 31711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:55.505720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:49465 (local address 127.11.8.126:38995)
0806 15:00:55.505812 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:55.505814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:55.505973 (+ 159us) server_negotiation.cc:407] Connection header received
0806 15:00:55.506719 (+ 746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:55.506720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:55.506732 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:55.506782 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:55.507762 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.508254 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.509309 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.509488 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.509541 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:55.511734 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:55.511735 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:55.511736 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:55.511741 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:55.512990 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:55.513405 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:55.513406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:55.513406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:55.513456 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:55.513694 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:55.513695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:55.513695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:55.513849 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:55.513932 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:55.514358 (+ 426us) server_negotiation.cc:299] Negotiation successful
0806 15:00:55.514471 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 15:00:55.514914 32104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:55.505669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.67:49465)
0806 15:00:55.505849 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:55.505886 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:55.506609 (+ 723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:55.506852 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:55.506856 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:55.507471 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:00:55.507678 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:55.507683 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.508335 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.508336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:55.509229 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:55.509231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.509561 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.509562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:55.509677 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:55.510160 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:00:55.510169 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:00:55.511638 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:00:55.513070 (+ 1432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:55.513077 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:55.513103 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:55.513337 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:55.513514 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:55.513515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:55.513515 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:55.513621 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:55.513933 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:00:55.513941 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:00:55.514141 (+ 200us) client_negotiation.cc:769] Sending connection context
0806 15:00:55.514317 (+ 176us) client_negotiation.cc:240] Negotiation successful
0806 15:00:55.514563 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":101,"threads_started":1}
I20260806 15:00:55.515470 32102 heartbeater.cc:347] Connected to a master server at 127.11.8.126:38995
I20260806 15:00:55.515561 32102 heartbeater.cc:464] Registering TS with master...
I20260806 15:00:55.515787 32102 heartbeater.cc:511] Master 127.11.8.126:38995 requested a full tablet report, sending...
I20260806 15:00:55.516185 31451 ts_manager.cc:194] Registered new tserver with Master: fb63490d5ac94a598651beff7b742c19 (127.11.8.67:38519)
I20260806 15:00:55.516582 31451 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.67@KRBTEST.COM'} at 127.11.8.67:49465
I20260806 15:00:55.520954 32107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:55.514261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47712 (local address 127.11.8.126:38995)
0806 15:00:55.514371 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:55.514373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:55.514596 (+ 223us) server_negotiation.cc:407] Connection header received
0806 15:00:55.514794 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:55.514795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:55.514807 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:55.514871 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:55.515750 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.516330 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.517134 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.517327 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.517419 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:55.518588 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:55.518590 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:55.518590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:55.518597 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:55.519840 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:55.520180 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:55.520181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:55.520181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:55.520251 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:55.520444 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:55.520445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:55.520445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:55.520551 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:55.520733 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:55.520772 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 15:00:55.520848 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":64,"threads_started":1}
I20260806 15:00:55.521696 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:00:55.524714 32107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:55.522041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47726 (local address 127.11.8.126:38995)
0806 15:00:55.522106 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:55.522109 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:55.522207 (+ 98us) server_negotiation.cc:407] Connection header received
0806 15:00:55.522293 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:55.522294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:55.522332 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:55.522418 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:00:55.522569 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.523075 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.523884 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.524107 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.524195 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:55.524381 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:00:55.524485 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:00:55.524583 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:55.524610 (+ 27us) server_negotiation.cc:299] Negotiation successful
0806 15:00:55.524632 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 15:00:55.524991 31451 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:47726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:00:55.526822 31451 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:00:55.529671 31616 tablet_service.cc:1467] Processing CreateTablet for tablet 2805bf10e7e64baaada70de1513be448 (DEFAULT_TABLE table=test-table [id=85f4182abf414d6a88857484726675b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:00:55.529899 31616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2805bf10e7e64baaada70de1513be448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:00:55.531332 31563 tablet_bootstrap.cc:492] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c: Bootstrap starting.
I20260806 15:00:55.531684 31563 tablet_bootstrap.cc:654] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c: Neither blocks nor log segments found. Creating new log.
I20260806 15:00:55.532595 31563 tablet_bootstrap.cc:492] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c: No bootstrap required, opened a new log
I20260806 15:00:55.532672 31563 ts_tablet_manager.cc:1397] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:00:55.533088 31563 raft_consensus.cc:348] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } }
I20260806 15:00:55.533186 31563 raft_consensus.cc:374] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:00:55.533248 31563 raft_consensus.cc:729] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39babaadc45f4e808c7cbb9b96805b7c, State: Initialized, Role: FOLLOWER
I20260806 15:00:55.533298 31563 consensus_queue.cc:260] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } }
I20260806 15:00:55.533409 31563 ts_tablet_manager.cc:1428] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:00:55.534021 31717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:55.529067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:44099 (local address 127.0.0.1:55744)
0806 15:00:55.529310 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:55.529320 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:55.529351 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:55.529888 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:55.529889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:55.529891 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:00:55.529930 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:55.529932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.531489 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.531490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:55.532836 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:55.532841 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.533743 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.533744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:55.533820 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:55.533848 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 15:00:55.533882 (+ 34us) client_negotiation.cc:240] Negotiation successful
0806 15:00:55.533934 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":43,"thread_start_us":110,"threads_started":1}
I20260806 15:00:55.534039 32111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:55.528947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:38519 (local address 127.0.0.1:47560)
0806 15:00:55.529231 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:55.529243 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:55.529293 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:55.529628 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:55.529629 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:55.529631 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:00:55.529681 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:55.529683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.531335 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.531336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:55.532816 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:55.532820 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.533828 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.533829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:55.533888 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:55.533891 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:00:55.533938 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 15:00:55.533969 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":27}
I20260806 15:00:55.534224 32112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:55.529095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47560 (local address 127.11.8.67:38519)
0806 15:00:55.529394 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:55.529398 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:55.529408 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:00:55.529445 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:55.529447 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:55.529487 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:55.529587 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:00:55.529851 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.531209 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.532918 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.533634 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.533722 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:55.533784 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:55.534080 (+ 296us) server_negotiation.cc:299] Negotiation successful
0806 15:00:55.534129 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":124,"threads_started":1}
I20260806 15:00:55.534544 32113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:55.529172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55744 (local address 127.11.8.66:44099)
0806 15:00:55.529569 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:55.529575 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:55.529586 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:00:55.529628 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:55.529630 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:55.529648 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:55.530015 (+ 367us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:00:55.530169 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.531373 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.532938 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.533634 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.534279 (+ 645us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:55.534358 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:55.534380 (+ 22us) server_negotiation.cc:299] Negotiation successful
0806 15:00:55.534460 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":100,"threads_started":1}
I20260806 15:00:55.535068 31998 tablet_service.cc:1467] Processing CreateTablet for tablet 2805bf10e7e64baaada70de1513be448 (DEFAULT_TABLE table=test-table [id=85f4182abf414d6a88857484726675b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:00:55.535207 31806 tablet_service.cc:1467] Processing CreateTablet for tablet 2805bf10e7e64baaada70de1513be448 (DEFAULT_TABLE table=test-table [id=85f4182abf414d6a88857484726675b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:00:55.535434 31998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2805bf10e7e64baaada70de1513be448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:00:55.535593 31806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2805bf10e7e64baaada70de1513be448. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:00:55.538335 32115 tablet_bootstrap.cc:492] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19: Bootstrap starting.
I20260806 15:00:55.538632 32116 tablet_bootstrap.cc:492] T 2805bf10e7e64baaada70de1513be448 P 4e9bb16103274469a57a60f4b12005fc: Bootstrap starting.
I20260806 15:00:55.539083 32115 tablet_bootstrap.cc:654] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19: Neither blocks nor log segments found. Creating new log.
I20260806 15:00:55.539270 32116 tablet_bootstrap.cc:654] T 2805bf10e7e64baaada70de1513be448 P 4e9bb16103274469a57a60f4b12005fc: Neither blocks nor log segments found. Creating new log.
I20260806 15:00:55.539757 32115 log.cc:824] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19: Log is configured to *not* fsync() on all Append() calls
I20260806 15:00:55.539893 32116 log.cc:824] T 2805bf10e7e64baaada70de1513be448 P 4e9bb16103274469a57a60f4b12005fc: Log is configured to *not* fsync() on all Append() calls
I20260806 15:00:55.540509 32115 tablet_bootstrap.cc:492] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19: No bootstrap required, opened a new log
I20260806 15:00:55.540596 32115 ts_tablet_manager.cc:1397] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 15:00:55.540692 32116 tablet_bootstrap.cc:492] T 2805bf10e7e64baaada70de1513be448 P 4e9bb16103274469a57a60f4b12005fc: No bootstrap required, opened a new log
I20260806 15:00:55.540768 32116 ts_tablet_manager.cc:1397] T 2805bf10e7e64baaada70de1513be448 P 4e9bb16103274469a57a60f4b12005fc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:00:55.541965 32115 raft_consensus.cc:348] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } }
I20260806 15:00:55.542116 32115 raft_consensus.cc:374] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:00:55.542163 32115 raft_consensus.cc:729] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb63490d5ac94a598651beff7b742c19, State: Initialized, Role: FOLLOWER
I20260806 15:00:55.542294 32115 consensus_queue.cc:260] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } }
I20260806 15:00:55.542325 32116 raft_consensus.cc:348] T 2805bf10e7e64baaada70de1513be448 P 4e9bb16103274469a57a60f4b12005fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } }
I20260806 15:00:55.542524 32116 raft_consensus.cc:374] T 2805bf10e7e64baaada70de1513be448 P 4e9bb16103274469a57a60f4b12005fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:00:55.542580 32116 raft_consensus.cc:729] T 2805bf10e7e64baaada70de1513be448 P 4e9bb16103274469a57a60f4b12005fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e9bb16103274469a57a60f4b12005fc, State: Initialized, Role: FOLLOWER
I20260806 15:00:55.542609 32115 ts_tablet_manager.cc:1428] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:00:55.542609 32102 heartbeater.cc:503] Master 127.11.8.126:38995 was elected leader, sending a full tablet report...
I20260806 15:00:55.542717 32116 consensus_queue.cc:260] T 2805bf10e7e64baaada70de1513be448 P 4e9bb16103274469a57a60f4b12005fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } }
I20260806 15:00:55.543040 32116 ts_tablet_manager.cc:1428] T 2805bf10e7e64baaada70de1513be448 P 4e9bb16103274469a57a60f4b12005fc: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 15:00:55.543051 31909 heartbeater.cc:503] Master 127.11.8.126:38995 was elected leader, sending a full tablet report...
W20260806 15:00:55.632563 31910 tablet.cc:2369] T 2805bf10e7e64baaada70de1513be448 P 4e9bb16103274469a57a60f4b12005fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:00:55.666947 32119 raft_consensus.cc:482] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:00:55.667184 32119 raft_consensus.cc:504] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } }
I20260806 15:00:55.667639 32119 leader_election.cc:288] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e9bb16103274469a57a60f4b12005fc (127.11.8.66:44099), 39babaadc45f4e808c7cbb9b96805b7c (127.11.8.65:34479)
I20260806 15:00:55.670647 32104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:55.667853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:44099 (local address 127.11.8.67:51593)
0806 15:00:55.667922 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:55.667936 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:55.667984 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:55.668236 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:55.668238 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:55.668240 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:00:55.668319 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:55.668321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.669001 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.669002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:55.669776 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:55.669778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.670358 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.670359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:55.670461 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:55.670464 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:00:55.670530 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 15:00:55.670571 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 15:00:55.670647 32113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:55.668001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:51593 (local address 127.11.8.66:44099)
0806 15:00:55.668060 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:55.668062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:55.668073 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:00:55.668107 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:55.668107 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:55.668120 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:55.668205 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:00:55.668401 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.668892 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.669880 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.670270 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.670358 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:55.670370 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:55.670549 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 15:00:55.670585 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 15:00:55.670845 32122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:55.667986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:34479 (local address 127.11.8.67:59445)
0806 15:00:55.668152 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:55.668181 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:55.668239 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:55.668416 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:55.668417 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:55.668419 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:00:55.668508 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:55.668510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.669183 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.669184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:55.669934 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:55.669936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.670599 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.670600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:55.670669 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:55.670673 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:00:55.670727 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 15:00:55.670769 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":105,"threads_started":1}
I20260806 15:00:55.670856 31718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:55.668092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:59445 (local address 127.11.8.65:34479)
0806 15:00:55.668135 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:55.668137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:55.668233 (+ 96us) server_negotiation.cc:407] Connection header received
0806 15:00:55.668295 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:55.668296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:55.668326 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:55.668412 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:00:55.668589 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.669091 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.670031 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.670515 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.670599 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:55.670694 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:55.670743 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 15:00:55.670792 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 15:00:55.671069 31843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2805bf10e7e64baaada70de1513be448" candidate_uuid: "fb63490d5ac94a598651beff7b742c19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e9bb16103274469a57a60f4b12005fc" is_pre_election: true
I20260806 15:00:55.671236 31843 raft_consensus.cc:2454] T 2805bf10e7e64baaada70de1513be448 P 4e9bb16103274469a57a60f4b12005fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb63490d5ac94a598651beff7b742c19 in term 0.
I20260806 15:00:55.671396 31650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2805bf10e7e64baaada70de1513be448" candidate_uuid: "fb63490d5ac94a598651beff7b742c19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39babaadc45f4e808c7cbb9b96805b7c" is_pre_election: true
I20260806 15:00:55.671491 31935 leader_election.cc:302] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e9bb16103274469a57a60f4b12005fc, fb63490d5ac94a598651beff7b742c19; no voters:
I20260806 15:00:55.671574 31650 raft_consensus.cc:2454] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb63490d5ac94a598651beff7b742c19 in term 0.
I20260806 15:00:55.671622 32119 raft_consensus.cc:2781] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:00:55.671681 32119 raft_consensus.cc:482] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:00:55.671723 32119 raft_consensus.cc:3037] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:00:55.672456 32119 raft_consensus.cc:504] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } }
I20260806 15:00:55.672564 32119 leader_election.cc:288] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19 [CANDIDATE]: Term 1 election: Requested vote from peers 4e9bb16103274469a57a60f4b12005fc (127.11.8.66:44099), 39babaadc45f4e808c7cbb9b96805b7c (127.11.8.65:34479)
I20260806 15:00:55.672773 31843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2805bf10e7e64baaada70de1513be448" candidate_uuid: "fb63490d5ac94a598651beff7b742c19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e9bb16103274469a57a60f4b12005fc"
I20260806 15:00:55.672824 31650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2805bf10e7e64baaada70de1513be448" candidate_uuid: "fb63490d5ac94a598651beff7b742c19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39babaadc45f4e808c7cbb9b96805b7c"
I20260806 15:00:55.672883 31843 raft_consensus.cc:3037] T 2805bf10e7e64baaada70de1513be448 P 4e9bb16103274469a57a60f4b12005fc [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:00:55.672890 31650 raft_consensus.cc:3037] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:00:55.673584 31650 raft_consensus.cc:2454] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb63490d5ac94a598651beff7b742c19 in term 1.
I20260806 15:00:55.673705 31843 raft_consensus.cc:2454] T 2805bf10e7e64baaada70de1513be448 P 4e9bb16103274469a57a60f4b12005fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb63490d5ac94a598651beff7b742c19 in term 1.
I20260806 15:00:55.673782 31935 leader_election.cc:302] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39babaadc45f4e808c7cbb9b96805b7c, fb63490d5ac94a598651beff7b742c19; no voters:
I20260806 15:00:55.673871 32119 raft_consensus.cc:2781] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:00:55.674036 32119 raft_consensus.cc:686] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19 [term 1 LEADER]: Becoming Leader. State: Replica: fb63490d5ac94a598651beff7b742c19, State: Running, Role: LEADER
I20260806 15:00:55.674173 32119 consensus_queue.cc:237] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } }
I20260806 15:00:55.675203 31449 catalog_manager.cc:5495] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb63490d5ac94a598651beff7b742c19 (127.11.8.67). New cstate: current_term: 1 leader_uuid: "fb63490d5ac94a598651beff7b742c19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } health_report { overall_health: UNKNOWN } } }
I20260806 15:00:55.717115 32112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:55.714641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47568 (local address 127.11.8.67:38519)
0806 15:00:55.714733 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:55.714735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:55.714743 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:00:55.714782 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:55.714783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:55.714806 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:55.714883 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:00:55.715023 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.715513 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.716273 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:55.716471 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:55.716549 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:55.716730 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:00:55.716834 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:00:55.716926 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:55.716972 (+ 46us) server_negotiation.cc:299] Negotiation successful
0806 15:00:55.716995 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 15:00:55.719225 31650 raft_consensus.cc:1261] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [term 1 FOLLOWER]: Refusing update from remote peer fb63490d5ac94a598651beff7b742c19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:00:55.719336 31843 raft_consensus.cc:1261] T 2805bf10e7e64baaada70de1513be448 P 4e9bb16103274469a57a60f4b12005fc [term 1 FOLLOWER]: Refusing update from remote peer fb63490d5ac94a598651beff7b742c19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:00:55.719653 32119 consensus_queue.cc:1035] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:00:55.719837 32123 consensus_queue.cc:1035] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:00:55.722003 32126 mvcc.cc:205] Tried to move back new op lower bound from 7315572554623016960 to 7315572554442231808. Current Snapshot: MvccSnapshot[applied={T|T < 7315572554623016960}]
I20260806 15:00:55.722223 32127 mvcc.cc:205] Tried to move back new op lower bound from 7315572554623016960 to 7315572554442231808. Current Snapshot: MvccSnapshot[applied={T|T < 7315572554623016960}]
I20260806 15:00:55.723026 32129 mvcc.cc:205] Tried to move back new op lower bound from 7315572554623016960 to 7315572554442231808. Current Snapshot: MvccSnapshot[applied={T|T < 7315572554623016960}]
I20260806 15:00:55.724423 31451 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:47726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:00:55.724519 31451 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:47726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:00:55.725277 31451 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30: Sending DeleteTablet for 3 replicas of tablet 2805bf10e7e64baaada70de1513be448
I20260806 15:00:55.725639 31806 tablet_service.cc:1514] Processing DeleteTablet for tablet 2805bf10e7e64baaada70de1513be448 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:00:55 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:55744
I20260806 15:00:55.725700 31998 tablet_service.cc:1514] Processing DeleteTablet for tablet 2805bf10e7e64baaada70de1513be448 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:00:55 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:47560
I20260806 15:00:55.725876 31616 tablet_service.cc:1514] Processing DeleteTablet for tablet 2805bf10e7e64baaada70de1513be448 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:00:55 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:52598
I20260806 15:00:55.725907 32140 tablet_replica.cc:330] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19: stopping tablet replica
I20260806 15:00:55.725968 31719 tablet_replica.cc:330] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c: stopping tablet replica
I20260806 15:00:55.725997 32140 raft_consensus.cc:2229] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19 [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:00:55.726018 31719 raft_consensus.cc:2229] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:00:55.726070 31719 raft_consensus.cc:2258] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:00:55.726096 32140 raft_consensus.cc:2258] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:00:55.726269 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 31509
I20260806 15:00:55.726598 32140 ts_tablet_manager.cc:1905] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:00:55.728891 32140 ts_tablet_manager.cc:1918] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:00:55.728973 32140 log.cc:1196] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/wal/wals/2805bf10e7e64baaada70de1513be448
I20260806 15:00:55.729414 32140 ts_tablet_manager.cc:1939] T 2805bf10e7e64baaada70de1513be448 P fb63490d5ac94a598651beff7b742c19: Deleting consensus metadata
I20260806 15:00:55.730821 31433 catalog_manager.cc:4841] TS fb63490d5ac94a598651beff7b742c19 (127.11.8.67:38519): tablet 2805bf10e7e64baaada70de1513be448 (table test-table [id=85f4182abf414d6a88857484726675b7]) successfully deleted
I20260806 15:00:55.738601 32139 tablet_replica.cc:330] T 2805bf10e7e64baaada70de1513be448 P 4e9bb16103274469a57a60f4b12005fc: stopping tablet replica
I20260806 15:00:55.738724 32139 raft_consensus.cc:2229] T 2805bf10e7e64baaada70de1513be448 P 4e9bb16103274469a57a60f4b12005fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:00:55.738818 32139 raft_consensus.cc:2258] T 2805bf10e7e64baaada70de1513be448 P 4e9bb16103274469a57a60f4b12005fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:00:55.739303 32139 ts_tablet_manager.cc:1905] T 2805bf10e7e64baaada70de1513be448 P 4e9bb16103274469a57a60f4b12005fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:00:55.740808 32139 ts_tablet_manager.cc:1918] T 2805bf10e7e64baaada70de1513be448 P 4e9bb16103274469a57a60f4b12005fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:00:55.740904 32139 log.cc:1196] T 2805bf10e7e64baaada70de1513be448 P 4e9bb16103274469a57a60f4b12005fc: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/wal/wals/2805bf10e7e64baaada70de1513be448
I20260806 15:00:55.741212 32139 ts_tablet_manager.cc:1939] T 2805bf10e7e64baaada70de1513be448 P 4e9bb16103274469a57a60f4b12005fc: Deleting consensus metadata
W20260806 15:00:55.741765 31432 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.65:34479 (error 108)
I20260806 15:00:55.742060 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 31715
I20260806 15:00:55.742069 31432 catalog_manager.cc:4841] TS 4e9bb16103274469a57a60f4b12005fc (127.11.8.66:44099): tablet 2805bf10e7e64baaada70de1513be448 (table test-table [id=85f4182abf414d6a88857484726675b7]) successfully deleted
I20260806 15:00:55.742434 32111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:55.742277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:34479 (local address 127.0.0.1:52600)
0806 15:00:55.742323 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:55.742370 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.65:34479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 15:00:55.742825 31432 catalog_manager.cc:4584] TS 39babaadc45f4e808c7cbb9b96805b7c (127.11.8.65:34479): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2805bf10e7e64baaada70de1513be448: Network error: Client connection negotiation failed: client connection to 127.11.8.65:34479: connect: Connection refused (error 111)
I20260806 15:00:55.752075 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 31912
I20260806 15:00:55.762575 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 31410
I20260806 15:00:55.772171 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:38995
--webserver_interface=127.11.8.126
--webserver_port=32903
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:37615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:38995
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:00:55.857657 32145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:00:55.857872 32145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:00:55.857896 32145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:00:55.859176 32145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:00:55.859225 32145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:00:55.859239 32145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:00:55.859254 32145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:00:55.859266 32145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:00:55.860826 32145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:37615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:38995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:38995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=32903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:00:55.861111 32145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:00:55.861358 32145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:00:55.866205 32150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:55.866190 32154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:55.866189 32151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:55.866570 32145 server_base.cc:1034] running on GCE node
I20260806 15:00:55.866773 32145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:00:55.867074 32145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:00:55.868255 32145 hybrid_clock.cc:648] HybridClock initialized: now 1786028455868244 us; error 29 us; skew 500 ppm
Aug 06 15:00:55 dist-test-slave-68xk krb5kdc[30611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028455, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:00:55.871202 32145 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 15:00:55.871701 32145 webserver.cc:460] Webserver started at http://127.11.8.126:32903/ using document root <none> and password file <none>
I20260806 15:00:55.871958 32145 fs_manager.cc:359] Metadata directory not provided
I20260806 15:00:55.872040 32145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:00:55.873441 32145 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 15:00:55.874155 32163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:55.874344 32145 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:00:55.874468 32145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "2f76c3cc03194bdcb237406279446e30"
format_stamp: "Formatted at 2026-08-06 15:00:54 on dist-test-slave-68xk"
server_key: "68b434bd754da26e7bed7757697f78fa"
server_key_iv: "e1ece243453028139ea624c9e6a98846"
server_key_version: "encryptionkey@0"
I20260806 15:00:55.874841 32145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:00:55.884268 32145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:00:55.886339 32145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:00:55.886540 32145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:00:55.890971 32145 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:38995
I20260806 15:00:55.891041 32235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:38995 every 8 connection(s)
I20260806 15:00:55.891438 32145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:00:55.893126 32236 sys_catalog.cc:263] Verifying existing consensus state
I20260806 15:00:55.893872 32236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30: Bootstrap starting.
I20260806 15:00:55.897761 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 32145
I20260806 15:00:55.898169 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:34479
--local_ip_for_outbound_sockets=127.11.8.65
--tserver_master_addrs=127.11.8.126:38995
--webserver_port=32789
--webserver_interface=127.11.8.65
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:37615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:00:55.904429 32236 log.cc:824] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30: Log is configured to *not* fsync() on all Append() calls
I20260806 15:00:55.907049 32236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 15:00:55.907413 32236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30: Bootstrap complete.
I20260806 15:00:55.909971 32236 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f76c3cc03194bdcb237406279446e30" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38995 } }
I20260806 15:00:55.910223 32236 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f76c3cc03194bdcb237406279446e30, State: Initialized, Role: FOLLOWER
I20260806 15:00:55.910450 32236 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f76c3cc03194bdcb237406279446e30" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38995 } }
I20260806 15:00:55.910538 32236 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:00:55.910594 32236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:00:55.910650 32236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [term 2 FOLLOWER]: Advancing to term 3
I20260806 15:00:55.911758 32236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f76c3cc03194bdcb237406279446e30" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38995 } }
I20260806 15:00:55.911882 32236 leader_election.cc:302] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f76c3cc03194bdcb237406279446e30; no voters:
I20260806 15:00:55.912127 32236 leader_election.cc:288] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 15:00:55.912168 32243 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 15:00:55.912299 32243 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [term 3 LEADER]: Becoming Leader. State: Replica: 2f76c3cc03194bdcb237406279446e30, State: Running, Role: LEADER
I20260806 15:00:55.912444 32236 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:00:55.912431 32243 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f76c3cc03194bdcb237406279446e30" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38995 } }
I20260806 15:00:55.912900 32243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f76c3cc03194bdcb237406279446e30. Latest consensus state: current_term: 3 leader_uuid: "2f76c3cc03194bdcb237406279446e30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f76c3cc03194bdcb237406279446e30" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38995 } } }
I20260806 15:00:55.912976 32243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [sys.catalog]: This master's current role is: LEADER
I20260806 15:00:55.913192 32244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "2f76c3cc03194bdcb237406279446e30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f76c3cc03194bdcb237406279446e30" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 38995 } } }
I20260806 15:00:55.913287 32244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f76c3cc03194bdcb237406279446e30 [sys.catalog]: This master's current role is: LEADER
I20260806 15:00:55.913203 32249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:00:55.915409 32249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:00:55.915511 32249 catalog_manager.cc:1259] Loaded cluster ID: f2ad979d149846c9a59bb665feff6281
I20260806 15:00:55.915545 32249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 15:00:55.915896 32249 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f76ebc499ad google::LogMessage::Fail() at ??:0
@ 0x7f76ebc4c88a google::LogMessage::SendToLog() at ??:0
@ 0x7f76ebc494de google::LogMessage::Flush() at ??:0
@ 0x7f76ebc4aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f76f3b6a0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f76f3b738b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f76edb170d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f76edb0c954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f76ee9706db start_thread at ??:0
@ 0x7f76ea74761f clone at ??:0
W20260806 15:00:56.000499 32238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:00:56.000727 32238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:00:56.000766 32238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:00:56.001909 32238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:00:56.001976 32238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:00:56.002022 32238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 15:00:56.003623 32238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:37615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:34479
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=32789
--tserver_master_addrs=127.11.8.126:38995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:00:56.003901 32238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:00:56.004186 32238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:00:56.009423 32284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:56.009438 32285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:56.009620 32238 server_base.cc:1034] running on GCE node
W20260806 15:00:56.009656 32288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:56.009874 32238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:00:56.010151 32238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:00:56.011325 32238 hybrid_clock.cc:648] HybridClock initialized: now 1786028456011316 us; error 30 us; skew 500 ppm
Aug 06 15:00:56 dist-test-slave-68xk krb5kdc[30611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028456, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:00:56.013850 32238 init.cc:377] Logged in from keytab as kudu/127.11.8.65@KRBTEST.COM (short username kudu)
I20260806 15:00:56.014389 32238 webserver.cc:460] Webserver started at http://127.11.8.65:32789/ using document root <none> and password file <none>
I20260806 15:00:56.014675 32238 fs_manager.cc:359] Metadata directory not provided
I20260806 15:00:56.014724 32238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:00:56.016064 32238 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:00:56.016794 32298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:56.016992 32238 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:00:56.017087 32238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "39babaadc45f4e808c7cbb9b96805b7c"
format_stamp: "Formatted at 2026-08-06 15:00:54 on dist-test-slave-68xk"
server_key: "a6b677c8b0f284677a2e52a57b42da2d"
server_key_iv: "3b66dac8d3c8158f34911f5d3118c833"
server_key_version: "encryptionkey@0"
I20260806 15:00:56.017441 32238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:00:56.033377 32238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:00:56.035689 32238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:00:56.035856 32238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:00:56.036095 32238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:00:56.036563 32307 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 15:00:56.038529 32238 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 15:00:56.038596 32238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 15:00:56.038640 32238 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 15:00:56.039296 32238 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 15:00:56.039350 32238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 15:00:56.039431 32307 tablet_bootstrap.cc:492] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c: Bootstrap starting.
I20260806 15:00:56.046913 32238 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:34479
I20260806 15:00:56.047031 32471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:34479 every 8 connection(s)
I20260806 15:00:56.047341 32238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
I20260806 15:00:56.050719 32307 log.cc:824] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c: Log is configured to *not* fsync() on all Append() calls
I20260806 15:00:56.053375 32307 tablet_bootstrap.cc:492] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 15:00:56.053695 32307 tablet_bootstrap.cc:492] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c: Bootstrap complete.
I20260806 15:00:56.054850 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 32238
I20260806 15:00:56.055027 32307 ts_tablet_manager.cc:1397] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.010s
I20260806 15:00:56.055136 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:44099
--local_ip_for_outbound_sockets=127.11.8.66
--tserver_master_addrs=127.11.8.126:38995
--webserver_port=43167
--webserver_interface=127.11.8.66
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:37615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:00:56.056470 32307 raft_consensus.cc:348] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } }
I20260806 15:00:56.056802 32307 raft_consensus.cc:729] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39babaadc45f4e808c7cbb9b96805b7c, State: Initialized, Role: FOLLOWER
I20260806 15:00:56.056962 32307 consensus_queue.cc:260] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } }
I20260806 15:00:56.057255 32307 ts_tablet_manager.cc:1428] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
W20260806 15:00:56.118736 32475 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:00:56.047996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.65:44653)
0806 15:00:56.048195 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:56.048239 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:56.048682 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:56.118497 (+ 69815us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":110,"threads_started":1}
W20260806 15:00:56.119009 32472 heartbeater.cc:644] Failed to heartbeat to 127.11.8.126:38995 (0 consecutive failures): Network error: Failed to ping master at 127.11.8.126:38995: Client connection negotiation failed: client connection to 127.11.8.126:38995: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 15:00:56.119349 32475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:56.119226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.65:46383)
0806 15:00:56.119284 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:56.119329 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 15:00:56.119717 32475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:56.119629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.65:55327)
0806 15:00:56.119677 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:56.119703 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 15:00:56.119870 32472 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 15:00:56.142681 32477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:00:56.142871 32477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:00:56.142895 32477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:00:56.144024 32477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:00:56.144065 32477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:00:56.144091 32477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 15:00:56.145526 32477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:37615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:44099
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=43167
--tserver_master_addrs=127.11.8.126:38995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:00:56.145762 32477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:00:56.145996 32477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:00:56.150933 32489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:56.151033 32484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:56.151255 32477 server_base.cc:1034] running on GCE node
W20260806 15:00:56.151300 32486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:56.151585 32477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:00:56.151849 32477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:00:56.153002 32477 hybrid_clock.cc:648] HybridClock initialized: now 1786028456152985 us; error 36 us; skew 500 ppm
Aug 06 15:00:56 dist-test-slave-68xk krb5kdc[30611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028456, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:00:56.155797 32477 init.cc:377] Logged in from keytab as kudu/127.11.8.66@KRBTEST.COM (short username kudu)
I20260806 15:00:56.156236 32477 webserver.cc:460] Webserver started at http://127.11.8.66:43167/ using document root <none> and password file <none>
I20260806 15:00:56.156418 32477 fs_manager.cc:359] Metadata directory not provided
I20260806 15:00:56.156458 32477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:00:56.157738 32477 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:00:56.158289 32497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:56.158504 32477 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 15:00:56.158591 32477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "4e9bb16103274469a57a60f4b12005fc"
format_stamp: "Formatted at 2026-08-06 15:00:54 on dist-test-slave-68xk"
server_key: "857015cc99ebfd6c1b16a1c5970bb958"
server_key_iv: "8ad05443322ec64461095a9ce287332a"
server_key_version: "encryptionkey@0"
I20260806 15:00:56.158886 32477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:00:56.168171 32477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:00:56.170024 32477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:00:56.170135 32477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:00:56.170310 32477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:00:56.170630 32477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:00:56.170665 32477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:56.170686 32477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:00:56.170733 32477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:56.176925 32477 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:44099
I20260806 15:00:56.177011 32658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:44099 every 8 connection(s)
I20260806 15:00:56.177341 32477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
I20260806 15:00:56.178376 32661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:56.177994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.66:55801)
0806 15:00:56.178160 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:56.178239 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
W20260806 15:00:56.178820 32659 heartbeater.cc:644] Failed to heartbeat to 127.11.8.126:38995 (0 consecutive failures): Network error: Failed to ping master at 127.11.8.126:38995: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
I20260806 15:00:56.179085 32661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:56.178996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.66:38929)
0806 15:00:56.179042 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:56.179068 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 15:00:56.179329 32661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:56.179254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.66:56523)
0806 15:00:56.179293 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:56.179315 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 15:00:56.179443 32659 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 15:00:56.181984 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 32477
I20260806 15:00:56.182233 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:38519
--local_ip_for_outbound_sockets=127.11.8.67
--tserver_master_addrs=127.11.8.126:38995
--webserver_port=40093
--webserver_interface=127.11.8.67
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:37615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:00:56.266069 32663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:00:56.266283 32663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:00:56.266306 32663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:00:56.267426 32663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:00:56.267465 32663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:00:56.267491 32663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 15:00:56.268919 32663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:37615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:38519
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=40093
--tserver_master_addrs=127.11.8.126:38995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:00:56.269110 32663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:00:56.269294 32663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:00:56.274344 32671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:56.274375 32672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:00:56.274354 32675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:00:56.274371 32663 server_base.cc:1034] running on GCE node
I20260806 15:00:56.274837 32663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:00:56.275130 32663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:00:56.276326 32663 hybrid_clock.cc:648] HybridClock initialized: now 1786028456276314 us; error 30 us; skew 500 ppm
Aug 06 15:00:56 dist-test-slave-68xk krb5kdc[30611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028456, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:00:56.278916 32663 init.cc:377] Logged in from keytab as kudu/127.11.8.67@KRBTEST.COM (short username kudu)
I20260806 15:00:56.279366 32663 webserver.cc:460] Webserver started at http://127.11.8.67:40093/ using document root <none> and password file <none>
I20260806 15:00:56.279629 32663 fs_manager.cc:359] Metadata directory not provided
I20260806 15:00:56.279706 32663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:00:56.281085 32663 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:00:56.281751 32683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:56.281929 32663 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:00:56.282043 32663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "fb63490d5ac94a598651beff7b742c19"
format_stamp: "Formatted at 2026-08-06 15:00:54 on dist-test-slave-68xk"
server_key: "20c41eeea84e4117abc8676dc2739c08"
server_key_iv: "003429adc2b2bd71ab645bf8539b877b"
server_key_version: "encryptionkey@0"
I20260806 15:00:56.282410 32663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:00:56.295234 32663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:00:56.297465 32663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:00:56.297653 32663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:00:56.297919 32663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:00:56.298247 32663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:00:56.298305 32663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:56.298348 32663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:00:56.298374 32663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:00:56.304634 32663 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:38519
I20260806 15:00:56.304688 375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:38519 every 8 connection(s)
I20260806 15:00:56.305048 32663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
I20260806 15:00:56.306139 379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:56.305758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.67:42495)
0806 15:00:56.305912 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:56.305998 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":72,"threads_started":1}
W20260806 15:00:56.306613 376 heartbeater.cc:644] Failed to heartbeat to 127.11.8.126:38995 (0 consecutive failures): Network error: Failed to ping master at 127.11.8.126:38995: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
I20260806 15:00:56.306938 379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:56.306846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.67:60587)
0806 15:00:56.306891 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:56.306919 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 15:00:56.307233 379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:56.307171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.67:52951)
0806 15:00:56.307202 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:56.307220 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260806 15:00:56.307363 376 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 15:00:56.307857 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 32663
I20260806 15:00:57.120585 381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:57.120324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.65:42785)
0806 15:00:57.120450 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:57.120489 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":60,"threads_started":1}
I20260806 15:00:57.179971 382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:57.179752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.66:32959)
0806 15:00:57.179886 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:57.179921 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":61,"threads_started":1}
I20260806 15:00:57.240259 383 raft_consensus.cc:482] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:00:57.240445 383 raft_consensus.cc:504] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } }
I20260806 15:00:57.240815 383 leader_election.cc:288] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb63490d5ac94a598651beff7b742c19 (127.11.8.67:38519), 4e9bb16103274469a57a60f4b12005fc (127.11.8.66:44099)
Aug 06 15:00:57 dist-test-slave-68xk krb5kdc[30611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028456, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for kudu/127.11.8.66@KRBTEST.COM
Aug 06 15:00:57 dist-test-slave-68xk krb5kdc[30611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028456, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for kudu/127.11.8.67@KRBTEST.COM
I20260806 15:00:57.252130 385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:57.241156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:36829 (local address 127.11.8.67:38519)
0806 15:00:57.241442 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:57.241452 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:57.241472 (+ 20us) server_negotiation.cc:407] Connection header received
0806 15:00:57.241923 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:57.241926 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:57.241979 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:57.242141 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:57.243001 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:57.244039 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:57.244886 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:57.245345 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:57.245479 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:57.248398 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:57.248402 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:57.248402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:57.248412 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:57.250500 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:57.250861 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:57.250864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:57.250864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:57.250941 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:57.251139 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:57.251140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:57.251140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:57.251413 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:57.251486 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:57.251892 (+ 406us) server_negotiation.cc:299] Negotiation successful
0806 15:00:57.252031 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":72,"threads_started":1}
I20260806 15:00:57.252153 389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:57.241269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:45177 (local address 127.11.8.66:44099)
0806 15:00:57.241427 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 15:00:57.241440 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 15:00:57.241474 (+ 34us) server_negotiation.cc:407] Connection header received
0806 15:00:57.241925 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:00:57.241927 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:00:57.241967 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:00:57.242122 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:00:57.242962 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:57.243918 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:57.244872 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:00:57.245290 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:00:57.245410 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:57.247933 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:00:57.247938 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:00:57.247938 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:00:57.247949 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:00:57.250105 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:57.250651 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:57.250654 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:57.250655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:57.250753 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:00:57.251017 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:00:57.251019 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:00:57.251019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:00:57.251351 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:00:57.251442 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 15:00:57.251892 (+ 450us) server_negotiation.cc:299] Negotiation successful
0806 15:00:57.252030 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":51,"threads_started":1}
I20260806 15:00:57.252168 384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:57.241075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:38519 (local address 127.11.8.65:36829)
0806 15:00:57.241232 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:57.241251 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:57.241314 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:57.242154 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:57.242156 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:57.242625 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:00:57.242829 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:57.242858 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:57.244153 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:57.244154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:57.244784 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:57.244786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:57.245442 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:57.245444 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:57.245544 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:57.246127 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:00:57.246137 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:00:57.248261 (+ 2124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:00:57.250577 (+ 2316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:57.250578 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:57.250579 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:57.250760 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:57.251027 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:57.251028 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:57.251028 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:57.251076 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:57.251497 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:00:57.251498 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:00:57.251674 (+ 176us) client_negotiation.cc:769] Sending connection context
0806 15:00:57.251809 (+ 135us) client_negotiation.cc:240] Negotiation successful
0806 15:00:57.252018 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":39,"thread_start_us":79,"threads_started":1}
I20260806 15:00:57.252420 381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:57.241032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:44099 (local address 127.11.8.65:45177)
0806 15:00:57.241297 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:57.241311 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:00:57.241380 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:00:57.242151 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:00:57.242154 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:00:57.242619 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:00:57.242830 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:57.242839 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:57.244026 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:57.244027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:57.244781 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:00:57.244783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:00:57.245396 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:00:57.245398 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:00:57.245531 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:00:57.246169 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:00:57.246174 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:00:57.247814 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:00:57.250228 (+ 2414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:57.250235 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:57.250237 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:57.250559 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:57.250825 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:00:57.250825 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:00:57.250826 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:00:57.250937 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:00:57.251484 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:00:57.251494 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:00:57.251674 (+ 180us) client_negotiation.cc:769] Sending connection context
0806 15:00:57.251807 (+ 133us) client_negotiation.cc:240] Negotiation successful
0806 15:00:57.252149 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":165}
W20260806 15:00:57.253453 32301 leader_election.cc:341] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4e9bb16103274469a57a60f4b12005fc (127.11.8.66:44099): Not found: Tablet not found: 2805bf10e7e64baaada70de1513be448
W20260806 15:00:57.253573 32302 leader_election.cc:341] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fb63490d5ac94a598651beff7b742c19 (127.11.8.67:38519): Not found: Tablet not found: 2805bf10e7e64baaada70de1513be448
I20260806 15:00:57.253638 32302 leader_election.cc:302] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39babaadc45f4e808c7cbb9b96805b7c; no voters: 4e9bb16103274469a57a60f4b12005fc, fb63490d5ac94a598651beff7b742c19
I20260806 15:00:57.253762 383 raft_consensus.cc:2726] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 15:00:57.307893 390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:57.307676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.67:46649)
0806 15:00:57.307807 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:57.307853 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":74,"threads_started":1}
I20260806 15:00:58.121353 391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:58.121113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.65:56913)
0806 15:00:58.121257 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:58.121300 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":69,"threads_started":1}
I20260806 15:00:58.180665 392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:58.180432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.66:41499)
0806 15:00:58.180576 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:58.180623 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":62,"threads_started":1}
I20260806 15:00:58.308542 394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:58.308313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.67:54973)
0806 15:00:58.308457 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:58.308500 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":63,"threads_started":1}
I20260806 15:00:59.029343 395 raft_consensus.cc:482] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:00:59.029438 395 raft_consensus.cc:504] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } }
I20260806 15:00:59.029601 395 leader_election.cc:288] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb63490d5ac94a598651beff7b742c19 (127.11.8.67:38519), 4e9bb16103274469a57a60f4b12005fc (127.11.8.66:44099)
W20260806 15:00:59.030129 32302 leader_election.cc:341] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fb63490d5ac94a598651beff7b742c19 (127.11.8.67:38519): Not found: Tablet not found: 2805bf10e7e64baaada70de1513be448
W20260806 15:00:59.030212 32301 leader_election.cc:341] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4e9bb16103274469a57a60f4b12005fc (127.11.8.66:44099): Not found: Tablet not found: 2805bf10e7e64baaada70de1513be448
I20260806 15:00:59.030268 32302 leader_election.cc:302] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39babaadc45f4e808c7cbb9b96805b7c; no voters: 4e9bb16103274469a57a60f4b12005fc, fb63490d5ac94a598651beff7b742c19
I20260806 15:00:59.030351 395 raft_consensus.cc:2726] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 15:00:59.122071 396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:59.121838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.65:44493)
0806 15:00:59.121986 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:59.122022 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":88,"threads_started":1}
I20260806 15:00:59.181407 397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:59.181188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.66:60197)
0806 15:00:59.181326 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:59.181367 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":61,"threads_started":1}
I20260806 15:00:59.309315 398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:00:59.309035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.67:51357)
0806 15:00:59.309199 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 15:00:59.309265 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":87,"threads_started":1}
I20260806 15:01:00.122913 400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:00.122644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.65:36715)
0806 15:01:00.122823 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:00.122864 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":103,"threads_started":1}
I20260806 15:01:00.182142 401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:00.181888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.66:42551)
0806 15:01:00.182020 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:00.182056 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":60,"threads_started":1}
I20260806 15:01:00.310143 403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:00.309896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.67:50835)
0806 15:01:00.310051 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:00.310094 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":69,"threads_started":1}
I20260806 15:01:00.810884 404 raft_consensus.cc:482] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:01:00.811019 404 raft_consensus.cc:504] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } }
I20260806 15:01:00.811252 404 leader_election.cc:288] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb63490d5ac94a598651beff7b742c19 (127.11.8.67:38519), 4e9bb16103274469a57a60f4b12005fc (127.11.8.66:44099)
W20260806 15:01:00.811800 32302 leader_election.cc:341] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fb63490d5ac94a598651beff7b742c19 (127.11.8.67:38519): Not found: Tablet not found: 2805bf10e7e64baaada70de1513be448
W20260806 15:01:00.811872 32301 leader_election.cc:341] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4e9bb16103274469a57a60f4b12005fc (127.11.8.66:44099): Not found: Tablet not found: 2805bf10e7e64baaada70de1513be448
I20260806 15:01:00.811951 32301 leader_election.cc:302] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39babaadc45f4e808c7cbb9b96805b7c; no voters: 4e9bb16103274469a57a60f4b12005fc, fb63490d5ac94a598651beff7b742c19
I20260806 15:01:00.812031 404 raft_consensus.cc:2726] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 15:01:01.123736 405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:01.123471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.65:38231)
0806 15:01:01.123636 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:01.123680 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":78,"threads_started":1}
I20260806 15:01:01.182977 406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:01.182724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.66:53957)
0806 15:01:01.182883 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:01.182923 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":66,"threads_started":1}
I20260806 15:01:01.310890 407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:01.310667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.67:37685)
0806 15:01:01.310809 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:01.310847 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":71,"threads_started":1}
I20260806 15:01:02.124483 408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:02.124265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.65:56397)
0806 15:01:02.124392 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:02.124435 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
I20260806 15:01:02.183665 411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:02.183460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.66:57339)
0806 15:01:02.183581 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:02.183620 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":63,"threads_started":1}
I20260806 15:01:02.311659 412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:02.311416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.67:57339)
0806 15:01:02.311564 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:02.311606 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":69,"threads_started":1}
I20260806 15:01:03.125231 415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:03.124948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.65:43485)
0806 15:01:03.125133 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:03.125179 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":103,"threads_started":1}
I20260806 15:01:03.184270 416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:03.184064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.66:57793)
0806 15:01:03.184186 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:03.184225 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":68,"threads_started":1}
I20260806 15:01:03.205085 417 raft_consensus.cc:482] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:01:03.205183 417 raft_consensus.cc:504] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } }
I20260806 15:01:03.205350 417 leader_election.cc:288] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb63490d5ac94a598651beff7b742c19 (127.11.8.67:38519), 4e9bb16103274469a57a60f4b12005fc (127.11.8.66:44099)
W20260806 15:01:03.205783 32301 leader_election.cc:341] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4e9bb16103274469a57a60f4b12005fc (127.11.8.66:44099): Not found: Tablet not found: 2805bf10e7e64baaada70de1513be448
W20260806 15:01:03.205881 32302 leader_election.cc:341] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fb63490d5ac94a598651beff7b742c19 (127.11.8.67:38519): Not found: Tablet not found: 2805bf10e7e64baaada70de1513be448
I20260806 15:01:03.205981 32302 leader_election.cc:302] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39babaadc45f4e808c7cbb9b96805b7c; no voters: 4e9bb16103274469a57a60f4b12005fc, fb63490d5ac94a598651beff7b742c19
I20260806 15:01:03.206099 417 raft_consensus.cc:2726] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 15:01:03.312393 418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:03.312125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.67:41203)
0806 15:01:03.312296 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:03.312334 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":114,"threads_started":1}
I20260806 15:01:04.126103 419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:04.125838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.65:47179)
0806 15:01:04.126013 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:04.126054 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":76,"threads_started":1}
I20260806 15:01:04.184949 423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:04.184734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.66:60181)
0806 15:01:04.184864 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:04.184903 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260806 15:01:04.313153 424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:04.312909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.67:36943)
0806 15:01:04.313068 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:04.313108 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":68,"threads_started":1}
I20260806 15:01:05.126914 426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:05.126685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.65:53707)
0806 15:01:05.126825 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:05.126866 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":70,"threads_started":1}
I20260806 15:01:05.185732 427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:05.185429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.66:54011)
0806 15:01:05.185621 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:05.185668 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":76,"threads_started":1}
I20260806 15:01:05.313925 428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:05.313662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.67:53239)
0806 15:01:05.313836 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:05.313876 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":99,"threads_started":1}
I20260806 15:01:06.127645 429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:06.127411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.65:54335)
0806 15:01:06.127550 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:06.127595 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":66,"threads_started":1}
I20260806 15:01:06.186512 431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:06.186297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.66:57501)
0806 15:01:06.186435 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:06.186470 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":65,"threads_started":1}
I20260806 15:01:06.314739 432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:06.314495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.67:34293)
0806 15:01:06.314647 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:06.314689 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":63,"threads_started":1}
I20260806 15:01:07.128319 433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:07.128107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.65:41199)
0806 15:01:07.128233 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:07.128270 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
I20260806 15:01:07.187161 434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:07.186970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.66:42739)
0806 15:01:07.187084 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:07.187118 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":60,"threads_started":1}
I20260806 15:01:07.315481 435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:07.315220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.67:53213)
0806 15:01:07.315383 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:07.315430 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":83,"threads_started":1}
I20260806 15:01:08.128954 436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:08.128737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.65:52029)
0806 15:01:08.128872 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:08.128907 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":64,"threads_started":1}
I20260806 15:01:08.188055 437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:08.187836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.66:59077)
0806 15:01:08.187966 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:08.188007 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":61,"threads_started":1}
I20260806 15:01:08.316246 438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:08.315987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.67:39183)
0806 15:01:08.316160 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:08.316200 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":74,"threads_started":1}
I20260806 15:01:09.129665 439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:09.129436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.65:53489)
0806 15:01:09.129579 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:09.129619 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":87,"threads_started":1}
I20260806 15:01:09.188791 440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:09.188560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.66:41917)
0806 15:01:09.188707 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:09.188743 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":87,"threads_started":1}
I20260806 15:01:09.317168 442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:09.316912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.67:43467)
0806 15:01:09.317082 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:09.317122 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":86,"threads_started":1}
I20260806 15:01:10.130328 443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:10.130093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.65:56185)
0806 15:01:10.130243 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:10.130282 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
I20260806 15:01:10.189520 445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:10.189296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.66:60345)
0806 15:01:10.189440 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:10.189475 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260806 15:01:10.242610 446 raft_consensus.cc:482] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:01:10.242707 446 raft_consensus.cc:504] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb63490d5ac94a598651beff7b742c19" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38519 } } peers { permanent_uuid: "4e9bb16103274469a57a60f4b12005fc" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44099 } } peers { permanent_uuid: "39babaadc45f4e808c7cbb9b96805b7c" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34479 } }
I20260806 15:01:10.242919 446 leader_election.cc:288] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb63490d5ac94a598651beff7b742c19 (127.11.8.67:38519), 4e9bb16103274469a57a60f4b12005fc (127.11.8.66:44099)
W20260806 15:01:10.243427 32301 leader_election.cc:341] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4e9bb16103274469a57a60f4b12005fc (127.11.8.66:44099): Not found: Tablet not found: 2805bf10e7e64baaada70de1513be448
W20260806 15:01:10.243520 32302 leader_election.cc:341] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fb63490d5ac94a598651beff7b742c19 (127.11.8.67:38519): Not found: Tablet not found: 2805bf10e7e64baaada70de1513be448
I20260806 15:01:10.243583 32302 leader_election.cc:302] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 39babaadc45f4e808c7cbb9b96805b7c; no voters: 4e9bb16103274469a57a60f4b12005fc, fb63490d5ac94a598651beff7b742c19
I20260806 15:01:10.243674 446 raft_consensus.cc:2726] T 2805bf10e7e64baaada70de1513be448 P 39babaadc45f4e808c7cbb9b96805b7c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 15:01:10.318024 449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:10.317775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.67:45701)
0806 15:01:10.317922 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:10.317970 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":70,"threads_started":1}
I20260806 15:01:11.131080 450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:11.130874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.65:42423)
0806 15:01:11.130999 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:11.131035 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":67,"threads_started":1}
I20260806 15:01:11.190341 451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:11.190103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.66:58357)
0806 15:01:11.190249 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:11.190289 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":80,"threads_started":1}
I20260806 15:01:11.312422 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 32238
I20260806 15:01:11.320595 453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:11.319577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:38995 (local address 127.11.8.67:55021)
0806 15:01:11.320482 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:11.320539 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:38995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":880,"thread_start_us":546,"threads_started":1}
I20260806 15:01:11.324893 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 32477
I20260806 15:01:11.337857 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 32663
2026-08-06T15:01:11Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18673 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:01:11 dist-test-slave-68xk krb5kdc[456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:01:11 dist-test-slave-68xk krb5kdc[456](info): set up 2 sockets
Aug 06 15:01:11 dist-test-slave-68xk krb5kdc[456](info): commencing operation
krb5kdc: starting...
W20260806 15:01:12.543577 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.181s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 15:01:12 dist-test-slave-68xk krb5kdc[456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:01:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:01:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 15:01:12.670599 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:40703
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:46555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:40703
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:01:12.754465 475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:01:12.754691 475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:01:12.754729 475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:01:12.755868 475 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 15:01:12.755930 475 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 15:01:12.755955 475 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 15:01:12.755985 475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:01:12.756014 475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:01:12.756042 475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:01:12.756067 475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:01:12.756101 475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:01:12.757581 475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:46555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:40703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:40703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:01:12.757851 475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:01:12.758076 475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:01:12.763114 483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:01:12.763550 487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:01:12.763595 475 server_base.cc:1034] running on GCE node
W20260806 15:01:12.763600 484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:01:12.763911 475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:01:12.764151 475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:01:12.765355 475 hybrid_clock.cc:648] HybridClock initialized: now 1786028472765345 us; error 42 us; skew 500 ppm
Aug 06 15:01:12 dist-test-slave-68xk krb5kdc[456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028472, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:01:12.767777 475 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 15:01:12.768217 475 webserver.cc:460] Webserver started at http://127.11.8.126:41273/ using document root <none> and password file <none>
I20260806 15:01:12.768466 475 fs_manager.cc:359] Metadata directory not provided
I20260806 15:01:12.768533 475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:01:12.768672 475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:01:12.769654 475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "3ee53d72c6d14537a17757e1ead96d56"
format_stamp: "Formatted at 2026-08-06 15:01:12 on dist-test-slave-68xk"
server_key: "28867198b781e661e2450bb4470137da"
server_key_iv: "d9693c96ace7a2708ed265bac6fe7abd"
server_key_version: "encryptionkey@0"
I20260806 15:01:12.769999 475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "3ee53d72c6d14537a17757e1ead96d56"
format_stamp: "Formatted at 2026-08-06 15:01:12 on dist-test-slave-68xk"
server_key: "28867198b781e661e2450bb4470137da"
server_key_iv: "d9693c96ace7a2708ed265bac6fe7abd"
server_key_version: "encryptionkey@0"
I20260806 15:01:12.771410 475 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:01:12.772194 500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:01:12.772390 475 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:01:12.772493 475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "3ee53d72c6d14537a17757e1ead96d56"
format_stamp: "Formatted at 2026-08-06 15:01:12 on dist-test-slave-68xk"
server_key: "28867198b781e661e2450bb4470137da"
server_key_iv: "d9693c96ace7a2708ed265bac6fe7abd"
server_key_version: "encryptionkey@0"
I20260806 15:01:12.772564 475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:01:12.791054 475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:01:12.793150 475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:01:12.793334 475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:01:12.797564 475 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:40703
I20260806 15:01:12.797605 577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:40703 every 8 connection(s)
I20260806 15:01:12.797983 475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:01:12.798933 578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:01:12.801240 578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56: Bootstrap starting.
I20260806 15:01:12.801789 578 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56: Neither blocks nor log segments found. Creating new log.
I20260806 15:01:12.802042 578 log.cc:824] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56: Log is configured to *not* fsync() on all Append() calls
I20260806 15:01:12.802742 578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56: No bootstrap required, opened a new log
I20260806 15:01:12.804263 578 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee53d72c6d14537a17757e1ead96d56" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40703 } }
I20260806 15:01:12.804399 578 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:01:12.804422 578 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ee53d72c6d14537a17757e1ead96d56, State: Initialized, Role: FOLLOWER
I20260806 15:01:12.804523 578 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee53d72c6d14537a17757e1ead96d56" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40703 } }
I20260806 15:01:12.804612 578 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:01:12.804661 578 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:01:12.804716 578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:01:12.805276 578 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee53d72c6d14537a17757e1ead96d56" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40703 } }
I20260806 15:01:12.805400 578 leader_election.cc:302] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ee53d72c6d14537a17757e1ead96d56; no voters:
I20260806 15:01:12.805584 578 leader_election.cc:288] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:01:12.805634 581 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:01:12.805841 581 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [term 1 LEADER]: Becoming Leader. State: Replica: 3ee53d72c6d14537a17757e1ead96d56, State: Running, Role: LEADER
I20260806 15:01:12.805857 578 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:01:12.806229 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 475
I20260806 15:01:12.805954 581 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee53d72c6d14537a17757e1ead96d56" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40703 } }
I20260806 15:01:12.806375 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:01:12.806622 11297 external_mini_cluster.cc:1428] Setting key 02ac5bb29dabcc4bc86f219e6d2b1df0
I20260806 15:01:12.807044 583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ee53d72c6d14537a17757e1ead96d56. Latest consensus state: current_term: 1 leader_uuid: "3ee53d72c6d14537a17757e1ead96d56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee53d72c6d14537a17757e1ead96d56" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40703 } } }
I20260806 15:01:12.807240 583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [sys.catalog]: This master's current role is: LEADER
I20260806 15:01:12.807637 582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ee53d72c6d14537a17757e1ead96d56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee53d72c6d14537a17757e1ead96d56" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40703 } } }
I20260806 15:01:12.807729 582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [sys.catalog]: This master's current role is: LEADER
W20260806 15:01:12.807921 611 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:01:12.808019 611 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:01:12.808146 609 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:01:12.808346 609 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:01:12.810036 609 catalog_manager.cc:1347] Generated new cluster ID: 4bab1b69d1be4c5490895f85a7fe86c8
I20260806 15:01:12.810096 609 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 15:01:12 dist-test-slave-68xk krb5kdc[456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:01:12.817793 612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:12.807442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44828 (local address 127.11.8.126:40703)
0806 15:01:12.807660 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 15:01:12.807669 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:01:12.807936 (+ 267us) server_negotiation.cc:407] Connection header received
0806 15:01:12.808068 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:01:12.808076 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:01:12.808149 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:01:12.808351 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:01:12.809084 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:12.810371 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:12.811201 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:12.811641 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:12.811770 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:12.814460 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:01:12.814465 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:01:12.814465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:01:12.814479 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:01:12.816184 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:01:12.816573 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:01:12.816577 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:01:12.816577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:01:12.816675 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:01:12.816898 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:01:12.816899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:01:12.816899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:01:12.817148 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:01:12.817298 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 15:01:12.817430 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 15:01:12.817576 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":76,"threads_started":1}
I20260806 15:01:12.823721 609 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:01:12.824366 609 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:01:12.834443 609 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56: Generated new TSK 0
I20260806 15:01:12.834656 609 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
I20260806 15:01:12.869027 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:0
--local_ip_for_outbound_sockets=127.11.8.65
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:40703
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:46555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:01:12.953504 620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:01:12.953745 620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:01:12.953787 620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:01:12.955046 620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:01:12.955116 620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:01:12.955157 620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 15:01:12.956625 620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:46555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:40703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:01:12.956902 620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:01:12.957134 620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:01:12.962182 626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:01:12.962163 627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:01:12.962158 629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:01:12.962445 620 server_base.cc:1034] running on GCE node
I20260806 15:01:12.962718 620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:01:12.962985 620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:01:12.964167 620 hybrid_clock.cc:648] HybridClock initialized: now 1786028472964162 us; error 38 us; skew 500 ppm
Aug 06 15:01:12 dist-test-slave-68xk krb5kdc[456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028472, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:01:12.966853 620 init.cc:377] Logged in from keytab as kudu/127.11.8.65@KRBTEST.COM (short username kudu)
I20260806 15:01:12.967290 620 webserver.cc:460] Webserver started at http://127.11.8.65:36227/ using document root <none> and password file <none>
I20260806 15:01:12.967526 620 fs_manager.cc:359] Metadata directory not provided
I20260806 15:01:12.967599 620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:01:12.967760 620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:01:12.968816 620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/data/instance:
uuid: "cc682dee81814201b314bd6b53858bf8"
format_stamp: "Formatted at 2026-08-06 15:01:12 on dist-test-slave-68xk"
server_key: "cfbd07e199a839815a67dda70e31f091"
server_key_iv: "21ebc6b703bac7c47c255ff9062d4e62"
server_key_version: "encryptionkey@0"
I20260806 15:01:12.969214 620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance:
uuid: "cc682dee81814201b314bd6b53858bf8"
format_stamp: "Formatted at 2026-08-06 15:01:12 on dist-test-slave-68xk"
server_key: "cfbd07e199a839815a67dda70e31f091"
server_key_iv: "21ebc6b703bac7c47c255ff9062d4e62"
server_key_version: "encryptionkey@0"
I20260806 15:01:12.970678 620 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 15:01:12.971418 640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:01:12.971596 620 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:01:12.971704 620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "cc682dee81814201b314bd6b53858bf8"
format_stamp: "Formatted at 2026-08-06 15:01:12 on dist-test-slave-68xk"
server_key: "cfbd07e199a839815a67dda70e31f091"
server_key_iv: "21ebc6b703bac7c47c255ff9062d4e62"
server_key_version: "encryptionkey@0"
I20260806 15:01:12.971781 620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:01:13.001125 620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:01:13.003216 620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:01:13.003384 620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:01:13.003634 620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:01:13.003964 620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:01:13.004038 620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:01:13.004105 620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:01:13.004137 620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:01:13.012194 620 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:41611
I20260806 15:01:13.012274 796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:41611 every 8 connection(s)
I20260806 15:01:13.012630 620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
I20260806 15:01:13.015393 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 620
I20260806 15:01:13.015515 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance
I20260806 15:01:13.015738 11297 external_mini_cluster.cc:1428] Setting key e5972dcbb38213ab704df78d241bdabb
Aug 06 15:01:13 dist-test-slave-68xk krb5kdc[456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028472, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:01:13.024494 612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:13.013828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:51037 (local address 127.11.8.126:40703)
0806 15:01:13.013897 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 15:01:13.013899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:01:13.013910 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:01:13.014567 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:01:13.014568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:01:13.014607 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:01:13.014683 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:01:13.015993 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.016633 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.018170 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.018532 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.018618 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:13.020847 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:01:13.020849 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:01:13.020849 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:01:13.020862 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:01:13.022954 (+ 2092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:01:13.023426 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:01:13.023427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:01:13.023428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:01:13.023496 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:01:13.023756 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:01:13.023758 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:01:13.023758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:01:13.023906 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:01:13.023985 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 15:01:13.024269 (+ 284us) server_negotiation.cc:299] Negotiation successful
0806 15:01:13.024383 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 15:01:13.024503 799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:13.013435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.65:51037)
0806 15:01:13.013625 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:13.013674 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 15:01:13.014355 (+ 681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:01:13.014780 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:01:13.014786 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:01:13.015636 (+ 850us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:01:13.015910 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:13.015916 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.016733 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.016734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:13.018054 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:13.018057 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.018623 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.018624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:13.018734 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:13.019269 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:01:13.019278 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:01:13.020714 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:01:13.023059 (+ 2345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:01:13.023066 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:01:13.023068 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:01:13.023331 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:01:13.023575 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:01:13.023575 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:01:13.023576 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:01:13.023684 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:01:13.023977 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:01:13.023982 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:01:13.024123 (+ 141us) client_negotiation.cc:769] Sending connection context
0806 15:01:13.024211 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 15:01:13.024333 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
I20260806 15:01:13.025153 797 heartbeater.cc:347] Connected to a master server at 127.11.8.126:40703
I20260806 15:01:13.025270 797 heartbeater.cc:464] Registering TS with master...
I20260806 15:01:13.025447 797 heartbeater.cc:511] Master 127.11.8.126:40703 requested a full tablet report, sending...
I20260806 15:01:13.026021 524 ts_manager.cc:194] Registered new tserver with Master: cc682dee81814201b314bd6b53858bf8 (127.11.8.65:41611)
WARNING: no policy specified for kudu/127.11.8.66@KRBTEST.COM; defaulting to no policy
I20260806 15:01:13.026759 524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.65@KRBTEST.COM'} at 127.11.8.65:51037
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
I20260806 15:01:13.046023 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:0
--local_ip_for_outbound_sockets=127.11.8.66
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:40703
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:46555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:01:13.129817 802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:01:13.130024 802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:01:13.130048 802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:01:13.131263 802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:01:13.131310 802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:01:13.131336 802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 15:01:13.132728 802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:46555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:40703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:01:13.132917 802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:01:13.133095 802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:01:13.138213 811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:01:13.138208 808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:01:13.138208 807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:01:13.138720 802 server_base.cc:1034] running on GCE node
I20260806 15:01:13.138931 802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:01:13.139194 802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:01:13.140370 802 hybrid_clock.cc:648] HybridClock initialized: now 1786028473140355 us; error 33 us; skew 500 ppm
Aug 06 15:01:13 dist-test-slave-68xk krb5kdc[456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028473, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:01:13.143072 802 init.cc:377] Logged in from keytab as kudu/127.11.8.66@KRBTEST.COM (short username kudu)
I20260806 15:01:13.143716 802 webserver.cc:460] Webserver started at http://127.11.8.66:39133/ using document root <none> and password file <none>
I20260806 15:01:13.143965 802 fs_manager.cc:359] Metadata directory not provided
I20260806 15:01:13.144034 802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:01:13.144183 802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:01:13.145280 802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/data/instance:
uuid: "5238714247d84a09bbc6bf5d5697baa0"
format_stamp: "Formatted at 2026-08-06 15:01:13 on dist-test-slave-68xk"
server_key: "148c5a7019382d3f584a0e9249abe8a5"
server_key_iv: "b01e7045ad700035fc477a6f6ff21e75"
server_key_version: "encryptionkey@0"
I20260806 15:01:13.145665 802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance:
uuid: "5238714247d84a09bbc6bf5d5697baa0"
format_stamp: "Formatted at 2026-08-06 15:01:13 on dist-test-slave-68xk"
server_key: "148c5a7019382d3f584a0e9249abe8a5"
server_key_iv: "b01e7045ad700035fc477a6f6ff21e75"
server_key_version: "encryptionkey@0"
I20260806 15:01:13.147218 802 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:01:13.148090 821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:01:13.148306 802 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:01:13.148419 802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "5238714247d84a09bbc6bf5d5697baa0"
format_stamp: "Formatted at 2026-08-06 15:01:13 on dist-test-slave-68xk"
server_key: "148c5a7019382d3f584a0e9249abe8a5"
server_key_iv: "b01e7045ad700035fc477a6f6ff21e75"
server_key_version: "encryptionkey@0"
I20260806 15:01:13.148499 802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:01:13.159724 802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:01:13.162199 802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:01:13.162366 802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:01:13.162673 802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:01:13.163025 802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:01:13.163065 802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:01:13.163110 802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:01:13.163138 802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:01:13.169242 802 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:45941
I20260806 15:01:13.169277 978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:45941 every 8 connection(s)
I20260806 15:01:13.169691 802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
I20260806 15:01:13.171499 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 802
I20260806 15:01:13.171609 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance
I20260806 15:01:13.171803 11297 external_mini_cluster.cc:1428] Setting key 3ea6705a33120715726024b86381c28f
Aug 06 15:01:13 dist-test-slave-68xk krb5kdc[456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028473, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:01:13.181118 612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:13.170509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:52459 (local address 127.11.8.126:40703)
0806 15:01:13.170570 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 15:01:13.170573 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:01:13.171047 (+ 474us) server_negotiation.cc:407] Connection header received
0806 15:01:13.171758 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:01:13.171759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:01:13.171774 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:01:13.171826 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:01:13.172926 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.173444 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.174802 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.175002 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.175056 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:13.177946 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:01:13.177948 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:01:13.177948 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:01:13.177957 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:01:13.179546 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:01:13.180017 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:01:13.180019 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:01:13.180020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:01:13.180097 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:01:13.180388 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:01:13.180389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:01:13.180389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:01:13.180515 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:01:13.180601 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 15:01:13.180908 (+ 307us) server_negotiation.cc:299] Negotiation successful
0806 15:01:13.181009 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
WARNING: no policy specified for kudu/127.11.8.67@KRBTEST.COM; defaulting to no policy
I20260806 15:01:13.181329 981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:13.170636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.66:52459)
0806 15:01:13.170853 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:13.170919 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 15:01:13.171651 (+ 732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:01:13.171898 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:01:13.171903 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:01:13.172638 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:01:13.172826 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:13.172831 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.173547 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.173548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:13.174715 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:13.174718 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.175090 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.175091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:13.175235 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:13.176015 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:01:13.176027 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:01:13.177821 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:01:13.179642 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:01:13.179650 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:01:13.179653 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:01:13.179904 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:01:13.180176 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:01:13.180177 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:01:13.180178 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:01:13.180303 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:01:13.180592 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:01:13.180600 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:01:13.180755 (+ 155us) client_negotiation.cc:769] Sending connection context
0806 15:01:13.180871 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 15:01:13.181058 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":109,"threads_started":1}
I20260806 15:01:13.181945 979 heartbeater.cc:347] Connected to a master server at 127.11.8.126:40703
I20260806 15:01:13.182075 979 heartbeater.cc:464] Registering TS with master...
I20260806 15:01:13.182304 979 heartbeater.cc:511] Master 127.11.8.126:40703 requested a full tablet report, sending...
I20260806 15:01:13.182786 524 ts_manager.cc:194] Registered new tserver with Master: 5238714247d84a09bbc6bf5d5697baa0 (127.11.8.66:45941)
I20260806 15:01:13.183197 524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.66@KRBTEST.COM'} at 127.11.8.66:52459
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
I20260806 15:01:13.200899 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:0
--local_ip_for_outbound_sockets=127.11.8.67
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:40703
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:46555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:01:13.286537 984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:01:13.286739 984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:01:13.286770 984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:01:13.287876 984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:01:13.287911 984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:01:13.287937 984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 15:01:13.289335 984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:46555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:40703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:01:13.289533 984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:01:13.289702 984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:01:13.294847 993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:01:13.294858 995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:01:13.294858 992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:01:13.295032 984 server_base.cc:1034] running on GCE node
I20260806 15:01:13.295332 984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:01:13.295573 984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:01:13.296744 984 hybrid_clock.cc:648] HybridClock initialized: now 1786028473296724 us; error 33 us; skew 500 ppm
Aug 06 15:01:13 dist-test-slave-68xk krb5kdc[456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028473, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:01:13.299239 984 init.cc:377] Logged in from keytab as kudu/127.11.8.67@KRBTEST.COM (short username kudu)
I20260806 15:01:13.299664 984 webserver.cc:460] Webserver started at http://127.11.8.67:32795/ using document root <none> and password file <none>
I20260806 15:01:13.299921 984 fs_manager.cc:359] Metadata directory not provided
I20260806 15:01:13.299991 984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:01:13.300148 984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:01:13.301165 984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/data/instance:
uuid: "28bc371cef284ca4986b189d590a6d7f"
format_stamp: "Formatted at 2026-08-06 15:01:13 on dist-test-slave-68xk"
server_key: "6a6b1956ee51862f4c712fb7150b77fd"
server_key_iv: "8b79200cb7a8a88cfa3cccd708196941"
server_key_version: "encryptionkey@0"
I20260806 15:01:13.301537 984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance:
uuid: "28bc371cef284ca4986b189d590a6d7f"
format_stamp: "Formatted at 2026-08-06 15:01:13 on dist-test-slave-68xk"
server_key: "6a6b1956ee51862f4c712fb7150b77fd"
server_key_iv: "8b79200cb7a8a88cfa3cccd708196941"
server_key_version: "encryptionkey@0"
I20260806 15:01:13.302990 984 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:01:13.303716 1006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:01:13.303892 984 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:01:13.304001 984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "28bc371cef284ca4986b189d590a6d7f"
format_stamp: "Formatted at 2026-08-06 15:01:13 on dist-test-slave-68xk"
server_key: "6a6b1956ee51862f4c712fb7150b77fd"
server_key_iv: "8b79200cb7a8a88cfa3cccd708196941"
server_key_version: "encryptionkey@0"
I20260806 15:01:13.304075 984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:01:13.322750 984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:01:13.324940 984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:01:13.325125 984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:01:13.325421 984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:01:13.325753 984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:01:13.325811 984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:01:13.325855 984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:01:13.325881 984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:01:13.332064 984 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:41739
I20260806 15:01:13.332120 1172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:41739 every 8 connection(s)
I20260806 15:01:13.332428 984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
I20260806 15:01:13.336369 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 984
I20260806 15:01:13.336450 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance
I20260806 15:01:13.336629 11297 external_mini_cluster.cc:1428] Setting key 4041337cc47bac05665b059d3f215dd7
Aug 06 15:01:13 dist-test-slave-68xk krb5kdc[456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028473, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:01:13.342871 612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:13.333197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:57159 (local address 127.11.8.126:40703)
0806 15:01:13.333258 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 15:01:13.333260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:01:13.333470 (+ 210us) server_negotiation.cc:407] Connection header received
0806 15:01:13.333981 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:01:13.333981 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:01:13.333990 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:01:13.334048 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:01:13.335289 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.335777 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.337180 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.337490 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.337611 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:13.339933 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:01:13.339935 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:01:13.339935 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:01:13.339944 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:01:13.341331 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:01:13.341762 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:01:13.341763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:01:13.341764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:01:13.341833 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:01:13.342127 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:01:13.342128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:01:13.342128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:01:13.342257 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:01:13.342339 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 15:01:13.342684 (+ 345us) server_negotiation.cc:299] Negotiation successful
0806 15:01:13.342772 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 15:01:13.343099 1177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:13.333143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.67:57159)
0806 15:01:13.333324 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:13.333370 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 15:01:13.333891 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:01:13.334106 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:01:13.334110 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:01:13.334982 (+ 872us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:01:13.335197 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:13.335203 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.335868 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.335869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:13.337075 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:13.337078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.337624 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.337625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:13.337776 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:13.338349 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:01:13.338358 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:01:13.339828 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:01:13.341427 (+ 1599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:01:13.341434 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:01:13.341437 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:01:13.341678 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:01:13.341917 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:01:13.341918 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:01:13.341918 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:01:13.342052 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:01:13.342343 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:01:13.342351 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:01:13.342539 (+ 188us) client_negotiation.cc:769] Sending connection context
0806 15:01:13.342659 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 15:01:13.342829 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":103,"threads_started":1}
I20260806 15:01:13.343629 1173 heartbeater.cc:347] Connected to a master server at 127.11.8.126:40703
I20260806 15:01:13.343751 1173 heartbeater.cc:464] Registering TS with master...
I20260806 15:01:13.343926 1173 heartbeater.cc:511] Master 127.11.8.126:40703 requested a full tablet report, sending...
I20260806 15:01:13.344316 524 ts_manager.cc:194] Registered new tserver with Master: 28bc371cef284ca4986b189d590a6d7f (127.11.8.67:41739)
I20260806 15:01:13.344712 524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.67@KRBTEST.COM'} at 127.11.8.67:57159
I20260806 15:01:13.348356 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:01:13.355509 612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:13.349682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44834 (local address 127.11.8.126:40703)
0806 15:01:13.349759 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 15:01:13.349761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:01:13.349833 (+ 72us) server_negotiation.cc:407] Connection header received
0806 15:01:13.349869 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:01:13.349870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:01:13.349877 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:01:13.349929 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:01:13.350493 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.350946 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.351727 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.351908 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.351966 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:13.353077 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:01:13.353106 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:01:13.353106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:01:13.353113 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:01:13.354526 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:01:13.354826 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:01:13.354827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:01:13.354827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:01:13.354876 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:01:13.355038 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:01:13.355039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:01:13.355039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:01:13.355198 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:01:13.355280 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 15:01:13.355365 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 15:01:13.355422 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 15:01:13.357019 524 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:01:13.358599 524 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:01:13.365355 1189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:13.361022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:41611 (local address 127.0.0.1:38574)
0806 15:01:13.361228 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:13.361272 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 15:01:13.361343 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:01:13.361626 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:01:13.361629 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:01:13.361632 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:01:13.361718 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:13.361720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.362924 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.362925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:13.364065 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:13.364068 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.364983 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.364984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:13.365104 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:13.365125 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 15:01:13.365216 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 15:01:13.365273 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":78,"threads_started":1}
I20260806 15:01:13.365382 1190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:13.361210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38574 (local address 127.11.8.65:41611)
0806 15:01:13.361409 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0806 15:01:13.361414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:01:13.361425 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:01:13.361469 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:01:13.361471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:01:13.361489 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:01:13.361585 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:01:13.361803 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.362814 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.364164 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.364892 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.364963 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:13.365031 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 15:01:13.365244 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 15:01:13.365288 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":75,"threads_started":1}
I20260806 15:01:13.365784 1194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:13.361495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55860 (local address 127.11.8.66:45941)
0806 15:01:13.361892 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0806 15:01:13.361897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:01:13.361907 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:01:13.361945 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:01:13.361947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:01:13.361961 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:01:13.362034 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:01:13.362194 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.363349 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.364644 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.365327 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.365396 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:13.365451 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 15:01:13.365655 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 15:01:13.365702 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":116,"threads_started":1}
I20260806 15:01:13.365849 1188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:13.361071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:41739 (local address 127.0.0.1:58286)
0806 15:01:13.361380 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:13.361389 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:01:13.361435 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:01:13.362315 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:01:13.362316 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:01:13.362318 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:01:13.362358 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:13.362360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.363616 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.363617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:13.364441 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:13.364444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.365559 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.365561 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:13.365627 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:13.365631 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:01:13.365681 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 15:01:13.365723 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":105,"threads_started":1}
I20260806 15:01:13.365734 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:13.361282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:45941 (local address 127.0.0.1:55860)
0806 15:01:13.361714 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:13.361725 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:01:13.361773 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:01:13.362055 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:01:13.362056 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:01:13.362058 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:01:13.362105 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:13.362107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.363464 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.363465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:13.364551 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:13.364554 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.365487 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.365489 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:13.365553 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:13.365556 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:01:13.365625 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 15:01:13.365668 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":8,"thread_start_us":72,"threads_started":1}
I20260806 15:01:13.366153 1191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:13.361174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58286 (local address 127.11.8.67:41739)
0806 15:01:13.362135 (+ 961us) server_negotiation.cc:206] Beginning negotiation
0806 15:01:13.362138 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:01:13.362167 (+ 29us) server_negotiation.cc:407] Connection header received
0806 15:01:13.362203 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:01:13.362205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:01:13.362220 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:01:13.362305 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:01:13.362445 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.363481 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.364621 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.365366 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.365442 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:13.365516 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 15:01:13.365786 (+ 270us) server_negotiation.cc:299] Negotiation successful
0806 15:01:13.365836 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":932,"thread_start_us":127,"threads_started":1}
I20260806 15:01:13.366633 711 tablet_service.cc:1467] Processing CreateTablet for tablet f681deaef4e44698b2e220d0e1df374c (DEFAULT_TABLE table=test-table [id=f30ee5b738ca43ff89fdc0faf49a0042]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:01:13.366798 891 tablet_service.cc:1467] Processing CreateTablet for tablet f681deaef4e44698b2e220d0e1df374c (DEFAULT_TABLE table=test-table [id=f30ee5b738ca43ff89fdc0faf49a0042]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:01:13.366864 1077 tablet_service.cc:1467] Processing CreateTablet for tablet f681deaef4e44698b2e220d0e1df374c (DEFAULT_TABLE table=test-table [id=f30ee5b738ca43ff89fdc0faf49a0042]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:01:13.367055 711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f681deaef4e44698b2e220d0e1df374c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:01:13.367131 891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f681deaef4e44698b2e220d0e1df374c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:01:13.367239 1077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f681deaef4e44698b2e220d0e1df374c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:01:13.369830 1195 tablet_bootstrap.cc:492] T f681deaef4e44698b2e220d0e1df374c P 28bc371cef284ca4986b189d590a6d7f: Bootstrap starting.
I20260806 15:01:13.370671 1195 tablet_bootstrap.cc:654] T f681deaef4e44698b2e220d0e1df374c P 28bc371cef284ca4986b189d590a6d7f: Neither blocks nor log segments found. Creating new log.
I20260806 15:01:13.370741 1197 tablet_bootstrap.cc:492] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0: Bootstrap starting.
I20260806 15:01:13.370968 1196 tablet_bootstrap.cc:492] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8: Bootstrap starting.
I20260806 15:01:13.371403 1195 log.cc:824] T f681deaef4e44698b2e220d0e1df374c P 28bc371cef284ca4986b189d590a6d7f: Log is configured to *not* fsync() on all Append() calls
I20260806 15:01:13.371687 1196 tablet_bootstrap.cc:654] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8: Neither blocks nor log segments found. Creating new log.
I20260806 15:01:13.372275 1195 tablet_bootstrap.cc:492] T f681deaef4e44698b2e220d0e1df374c P 28bc371cef284ca4986b189d590a6d7f: No bootstrap required, opened a new log
I20260806 15:01:13.372252 1196 log.cc:824] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8: Log is configured to *not* fsync() on all Append() calls
I20260806 15:01:13.372346 1195 ts_tablet_manager.cc:1397] T f681deaef4e44698b2e220d0e1df374c P 28bc371cef284ca4986b189d590a6d7f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 15:01:13.373013 1196 tablet_bootstrap.cc:492] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8: No bootstrap required, opened a new log
I20260806 15:01:13.373075 1197 tablet_bootstrap.cc:654] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0: Neither blocks nor log segments found. Creating new log.
I20260806 15:01:13.373112 1196 ts_tablet_manager.cc:1397] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:01:13.373993 1197 log.cc:824] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0: Log is configured to *not* fsync() on all Append() calls
I20260806 15:01:13.374589 1195 raft_consensus.cc:348] T f681deaef4e44698b2e220d0e1df374c P 28bc371cef284ca4986b189d590a6d7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 } } peers { permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 } } peers { permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 } }
I20260806 15:01:13.374800 1195 raft_consensus.cc:374] T f681deaef4e44698b2e220d0e1df374c P 28bc371cef284ca4986b189d590a6d7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:01:13.374841 1195 raft_consensus.cc:729] T f681deaef4e44698b2e220d0e1df374c P 28bc371cef284ca4986b189d590a6d7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc371cef284ca4986b189d590a6d7f, State: Initialized, Role: FOLLOWER
I20260806 15:01:13.374943 1197 tablet_bootstrap.cc:492] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0: No bootstrap required, opened a new log
I20260806 15:01:13.375015 1195 consensus_queue.cc:260] T f681deaef4e44698b2e220d0e1df374c P 28bc371cef284ca4986b189d590a6d7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 } } peers { permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 } } peers { permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 } }
I20260806 15:01:13.375077 1197 ts_tablet_manager.cc:1397] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 15:01:13.375312 1195 ts_tablet_manager.cc:1428] T f681deaef4e44698b2e220d0e1df374c P 28bc371cef284ca4986b189d590a6d7f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 15:01:13.375355 1173 heartbeater.cc:503] Master 127.11.8.126:40703 was elected leader, sending a full tablet report...
I20260806 15:01:13.375300 1196 raft_consensus.cc:348] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 } } peers { permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 } } peers { permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 } }
I20260806 15:01:13.375475 1196 raft_consensus.cc:374] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:01:13.375512 1196 raft_consensus.cc:729] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc682dee81814201b314bd6b53858bf8, State: Initialized, Role: FOLLOWER
I20260806 15:01:13.375626 1196 consensus_queue.cc:260] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 } } peers { permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 } } peers { permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 } }
I20260806 15:01:13.375926 1196 ts_tablet_manager.cc:1428] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 15:01:13.376116 797 heartbeater.cc:503] Master 127.11.8.126:40703 was elected leader, sending a full tablet report...
I20260806 15:01:13.377233 1197 raft_consensus.cc:348] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 } } peers { permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 } } peers { permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 } }
I20260806 15:01:13.377374 1197 raft_consensus.cc:374] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:01:13.377395 1197 raft_consensus.cc:729] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5238714247d84a09bbc6bf5d5697baa0, State: Initialized, Role: FOLLOWER
I20260806 15:01:13.377519 1197 consensus_queue.cc:260] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 } } peers { permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 } } peers { permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 } }
I20260806 15:01:13.377846 979 heartbeater.cc:503] Master 127.11.8.126:40703 was elected leader, sending a full tablet report...
I20260806 15:01:13.377883 1197 ts_tablet_manager.cc:1428] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260806 15:01:13.420297 980 tablet.cc:2369] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:01:13.425876 1209 raft_consensus.cc:482] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:01:13.426011 1209 raft_consensus.cc:504] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 } } peers { permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 } } peers { permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 } }
I20260806 15:01:13.426440 1209 leader_election.cc:288] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc682dee81814201b314bd6b53858bf8 (127.11.8.65:41611), 28bc371cef284ca4986b189d590a6d7f (127.11.8.67:41739)
I20260806 15:01:13.430303 1191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:13.426712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:44997 (local address 127.11.8.67:41739)
0806 15:01:13.426755 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 15:01:13.426757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:01:13.426980 (+ 223us) server_negotiation.cc:407] Connection header received
0806 15:01:13.427007 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:01:13.427008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:01:13.427018 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:01:13.427083 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:01:13.427254 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.427938 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.429306 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.429893 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.429971 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:13.429981 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 15:01:13.430185 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 15:01:13.430231 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 15:01:13.430303 1190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:13.426687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:42479 (local address 127.11.8.65:41611)
0806 15:01:13.426740 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 15:01:13.426742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:01:13.426753 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:01:13.426810 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:01:13.426811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:01:13.426824 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:01:13.426881 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:01:13.427075 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.427845 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.429306 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.429893 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.429971 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:13.429982 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 15:01:13.430185 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 15:01:13.430241 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 15:01:13.430313 981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:13.426629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:41611 (local address 127.11.8.66:42479)
0806 15:01:13.426687 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:13.426700 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:01:13.426745 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:01:13.426913 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:01:13.426914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:01:13.426916 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:01:13.426993 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:13.426995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.427928 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.427929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:13.429202 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:13.429206 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.429986 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.429987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:13.430078 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:13.430082 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:01:13.430146 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 15:01:13.430194 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":5}
I20260806 15:01:13.430320 1210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:13.426638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:41739 (local address 127.11.8.66:44997)
0806 15:01:13.426850 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:13.426858 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 15:01:13.426934 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:01:13.427128 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:01:13.427128 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:01:13.427131 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:01:13.427178 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:13.427179 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.428012 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.428012 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:13.429217 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:13.429219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.429989 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.429989 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:13.430106 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:13.430110 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:01:13.430163 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 15:01:13.430196 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":40,"thread_start_us":152,"threads_started":1}
I20260806 15:01:13.430725 1105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f681deaef4e44698b2e220d0e1df374c" candidate_uuid: "5238714247d84a09bbc6bf5d5697baa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc371cef284ca4986b189d590a6d7f" is_pre_election: true
I20260806 15:01:13.430725 744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f681deaef4e44698b2e220d0e1df374c" candidate_uuid: "5238714247d84a09bbc6bf5d5697baa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc682dee81814201b314bd6b53858bf8" is_pre_election: true
I20260806 15:01:13.430888 1105 raft_consensus.cc:2454] T f681deaef4e44698b2e220d0e1df374c P 28bc371cef284ca4986b189d590a6d7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5238714247d84a09bbc6bf5d5697baa0 in term 0.
I20260806 15:01:13.430888 744 raft_consensus.cc:2454] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5238714247d84a09bbc6bf5d5697baa0 in term 0.
I20260806 15:01:13.431160 824 leader_election.cc:302] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc371cef284ca4986b189d590a6d7f, 5238714247d84a09bbc6bf5d5697baa0; no voters:
I20260806 15:01:13.431329 1209 raft_consensus.cc:2781] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:01:13.431409 1209 raft_consensus.cc:482] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:01:13.431429 1209 raft_consensus.cc:3037] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:01:13.432171 1209 raft_consensus.cc:504] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 } } peers { permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 } } peers { permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 } }
I20260806 15:01:13.432322 1209 leader_election.cc:288] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0 [CANDIDATE]: Term 1 election: Requested vote from peers cc682dee81814201b314bd6b53858bf8 (127.11.8.65:41611), 28bc371cef284ca4986b189d590a6d7f (127.11.8.67:41739)
I20260806 15:01:13.432540 1105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f681deaef4e44698b2e220d0e1df374c" candidate_uuid: "5238714247d84a09bbc6bf5d5697baa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc371cef284ca4986b189d590a6d7f"
I20260806 15:01:13.432538 744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f681deaef4e44698b2e220d0e1df374c" candidate_uuid: "5238714247d84a09bbc6bf5d5697baa0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc682dee81814201b314bd6b53858bf8"
I20260806 15:01:13.432610 1105 raft_consensus.cc:3037] T f681deaef4e44698b2e220d0e1df374c P 28bc371cef284ca4986b189d590a6d7f [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:01:13.432636 744 raft_consensus.cc:3037] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:01:13.433323 1105 raft_consensus.cc:2454] T f681deaef4e44698b2e220d0e1df374c P 28bc371cef284ca4986b189d590a6d7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5238714247d84a09bbc6bf5d5697baa0 in term 1.
I20260806 15:01:13.433452 744 raft_consensus.cc:2454] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5238714247d84a09bbc6bf5d5697baa0 in term 1.
I20260806 15:01:13.433493 824 leader_election.cc:302] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc371cef284ca4986b189d590a6d7f, 5238714247d84a09bbc6bf5d5697baa0; no voters:
I20260806 15:01:13.433624 1209 raft_consensus.cc:2781] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:01:13.433768 1209 raft_consensus.cc:686] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0 [term 1 LEADER]: Becoming Leader. State: Replica: 5238714247d84a09bbc6bf5d5697baa0, State: Running, Role: LEADER
I20260806 15:01:13.433868 1209 consensus_queue.cc:237] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 } } peers { permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 } } peers { permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 } }
I20260806 15:01:13.434741 523 catalog_manager.cc:5495] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5238714247d84a09bbc6bf5d5697baa0 (127.11.8.66). New cstate: current_term: 1 leader_uuid: "5238714247d84a09bbc6bf5d5697baa0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 } health_report { overall_health: HEALTHY } } }
I20260806 15:01:13.458396 1194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:13.455888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55866 (local address 127.11.8.66:45941)
0806 15:01:13.455972 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 15:01:13.455974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:01:13.455984 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:01:13.456032 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:01:13.456033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:01:13.456057 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:01:13.456139 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:01:13.456324 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.456856 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.457645 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:13.457845 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:13.457907 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:13.458079 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:01:13.458180 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:01:13.458245 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 15:01:13.458300 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 15:01:13.458318 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 15:01:13.460341 744 raft_consensus.cc:1261] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8 [term 1 FOLLOWER]: Refusing update from remote peer 5238714247d84a09bbc6bf5d5697baa0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:01:13.460350 1105 raft_consensus.cc:1261] T f681deaef4e44698b2e220d0e1df374c P 28bc371cef284ca4986b189d590a6d7f [term 1 FOLLOWER]: Refusing update from remote peer 5238714247d84a09bbc6bf5d5697baa0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:01:13.460716 1211 consensus_queue.cc:1035] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:01:13.460860 1209 consensus_queue.cc:1035] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:01:13.464056 1219 mvcc.cc:205] Tried to move back new op lower bound from 7315572627290832896 to 7315572627185696768. Current Snapshot: MvccSnapshot[applied={T|T < 7315572627290832896}]
I20260806 15:01:13.464334 1221 mvcc.cc:205] Tried to move back new op lower bound from 7315572627290832896 to 7315572627185696768. Current Snapshot: MvccSnapshot[applied={T|T < 7315572627290832896 or (T in {7315572627290832896})}]
I20260806 15:01:13.465516 524 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:01:13.465608 524 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:01:13.466394 524 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56: Sending DeleteTablet for 3 replicas of tablet f681deaef4e44698b2e220d0e1df374c
I20260806 15:01:13.466743 711 tablet_service.cc:1514] Processing DeleteTablet for tablet f681deaef4e44698b2e220d0e1df374c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:01:13 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:38574
I20260806 15:01:13.466823 1077 tablet_service.cc:1514] Processing DeleteTablet for tablet f681deaef4e44698b2e220d0e1df374c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:01:13 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:58286
I20260806 15:01:13.466850 891 tablet_service.cc:1514] Processing DeleteTablet for tablet f681deaef4e44698b2e220d0e1df374c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:01:13 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:55860
I20260806 15:01:13.467135 1235 tablet_replica.cc:330] T f681deaef4e44698b2e220d0e1df374c P 28bc371cef284ca4986b189d590a6d7f: stopping tablet replica
I20260806 15:01:13.467168 1233 tablet_replica.cc:330] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8: stopping tablet replica
I20260806 15:01:13.467167 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 620
I20260806 15:01:13.467288 1235 raft_consensus.cc:2229] T f681deaef4e44698b2e220d0e1df374c P 28bc371cef284ca4986b189d590a6d7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:01:13.467409 1235 raft_consensus.cc:2258] T f681deaef4e44698b2e220d0e1df374c P 28bc371cef284ca4986b189d590a6d7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:01:13.472255 1235 ts_tablet_manager.cc:1905] T f681deaef4e44698b2e220d0e1df374c P 28bc371cef284ca4986b189d590a6d7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:01:13.474093 1235 ts_tablet_manager.cc:1918] T f681deaef4e44698b2e220d0e1df374c P 28bc371cef284ca4986b189d590a6d7f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:01:13.474177 1235 log.cc:1196] T f681deaef4e44698b2e220d0e1df374c P 28bc371cef284ca4986b189d590a6d7f: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/wal/wals/f681deaef4e44698b2e220d0e1df374c
I20260806 15:01:13.474557 1235 ts_tablet_manager.cc:1939] T f681deaef4e44698b2e220d0e1df374c P 28bc371cef284ca4986b189d590a6d7f: Deleting consensus metadata
I20260806 15:01:13.474562 1234 tablet_replica.cc:330] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0: stopping tablet replica
I20260806 15:01:13.474668 1234 raft_consensus.cc:2229] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0 [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:01:13.474921 1234 raft_consensus.cc:2258] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:01:13.475137 505 catalog_manager.cc:4841] TS 28bc371cef284ca4986b189d590a6d7f (127.11.8.67:41739): tablet f681deaef4e44698b2e220d0e1df374c (table test-table [id=f30ee5b738ca43ff89fdc0faf49a0042]) successfully deleted
I20260806 15:01:13.475445 1234 ts_tablet_manager.cc:1905] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:01:13.477514 1234 ts_tablet_manager.cc:1918] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:01:13.477598 1234 log.cc:1196] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/wal/wals/f681deaef4e44698b2e220d0e1df374c
I20260806 15:01:13.477929 1234 ts_tablet_manager.cc:1939] T f681deaef4e44698b2e220d0e1df374c P 5238714247d84a09bbc6bf5d5697baa0: Deleting consensus metadata
I20260806 15:01:13.478351 505 catalog_manager.cc:4841] TS 5238714247d84a09bbc6bf5d5697baa0 (127.11.8.66:45941): tablet f681deaef4e44698b2e220d0e1df374c (table test-table [id=f30ee5b738ca43ff89fdc0faf49a0042]) successfully deleted
W20260806 15:01:13.479504 507 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.65:41611 (error 108)
I20260806 15:01:13.479738 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 802
I20260806 15:01:13.491016 1189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:13.490845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:41611 (local address 127.0.0.1:38580)
0806 15:01:13.490917 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:13.490968 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.65:41611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 15:01:13.491169 507 catalog_manager.cc:4584] TS cc682dee81814201b314bd6b53858bf8 (127.11.8.65:41611): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f681deaef4e44698b2e220d0e1df374c: Network error: Client connection negotiation failed: client connection to 127.11.8.65:41611: connect: Connection refused (error 111)
I20260806 15:01:13.494875 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 984
I20260806 15:01:13.503419 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 475
I20260806 15:01:33.512681 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:40703
--webserver_interface=127.11.8.126
--webserver_port=41273
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:46555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:40703
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:01:33.600225 1238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:01:33.600449 1238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:01:33.600488 1238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:01:33.601856 1238 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 15:01:33.601918 1238 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 15:01:33.601941 1238 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 15:01:33.601971 1238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:01:33.602000 1238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:01:33.602027 1238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:01:33.602056 1238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:01:33.602085 1238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:01:33.603799 1238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:46555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:40703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:40703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=41273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:01:33.604079 1238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:01:33.604323 1238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:01:33.613893 1247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:01:33.614019 1238 server_base.cc:1034] running on GCE node
W20260806 15:01:33.613893 1248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:01:33.614204 1250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:01:33.614548 1238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:01:33.614931 1238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:01:33.616110 1238 hybrid_clock.cc:648] HybridClock initialized: now 1786028493616103 us; error 39 us; skew 500 ppm
Aug 06 15:01:33 dist-test-slave-68xk krb5kdc[456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028493, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:01:33.618969 1238 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 15:01:33.619458 1238 webserver.cc:460] Webserver started at http://127.11.8.126:41273/ using document root <none> and password file <none>
I20260806 15:01:33.619724 1238 fs_manager.cc:359] Metadata directory not provided
I20260806 15:01:33.619817 1238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:01:33.621325 1238 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 15:01:33.622028 1263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:01:33.622222 1238 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 15:01:33.622344 1238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "3ee53d72c6d14537a17757e1ead96d56"
format_stamp: "Formatted at 2026-08-06 15:01:12 on dist-test-slave-68xk"
server_key: "28867198b781e661e2450bb4470137da"
server_key_iv: "d9693c96ace7a2708ed265bac6fe7abd"
server_key_version: "encryptionkey@0"
I20260806 15:01:33.622838 1238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:01:33.640810 1238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:01:33.643427 1238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:01:33.643605 1238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:01:33.648128 1238 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:40703
I20260806 15:01:33.648171 1333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:40703 every 8 connection(s)
I20260806 15:01:33.648612 1238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:01:33.648751 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 1238
I20260806 15:01:33.649158 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:41611
--local_ip_for_outbound_sockets=127.11.8.65
--tserver_master_addrs=127.11.8.126:40703
--webserver_port=36227
--webserver_interface=127.11.8.65
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:46555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:01:33.651067 1334 sys_catalog.cc:263] Verifying existing consensus state
I20260806 15:01:33.652097 1334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56: Bootstrap starting.
I20260806 15:01:33.661170 1334 log.cc:824] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56: Log is configured to *not* fsync() on all Append() calls
I20260806 15:01:33.663225 1334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 15:01:33.663533 1334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56: Bootstrap complete.
I20260806 15:01:33.665915 1334 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee53d72c6d14537a17757e1ead96d56" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40703 } }
I20260806 15:01:33.666163 1334 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ee53d72c6d14537a17757e1ead96d56, State: Initialized, Role: FOLLOWER
I20260806 15:01:33.666350 1334 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee53d72c6d14537a17757e1ead96d56" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40703 } }
I20260806 15:01:33.666452 1334 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:01:33.666515 1334 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:01:33.666565 1334 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [term 1 FOLLOWER]: Advancing to term 2
I20260806 15:01:33.667563 1334 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee53d72c6d14537a17757e1ead96d56" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40703 } }
I20260806 15:01:33.667689 1334 leader_election.cc:302] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ee53d72c6d14537a17757e1ead96d56; no voters:
I20260806 15:01:33.667896 1334 leader_election.cc:288] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 15:01:33.667932 1338 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 15:01:33.668061 1338 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [term 2 LEADER]: Becoming Leader. State: Replica: 3ee53d72c6d14537a17757e1ead96d56, State: Running, Role: LEADER
I20260806 15:01:33.668171 1334 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:01:33.668155 1338 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee53d72c6d14537a17757e1ead96d56" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40703 } }
I20260806 15:01:33.668435 1338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ee53d72c6d14537a17757e1ead96d56. Latest consensus state: current_term: 2 leader_uuid: "3ee53d72c6d14537a17757e1ead96d56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee53d72c6d14537a17757e1ead96d56" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40703 } } }
I20260806 15:01:33.668506 1338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [sys.catalog]: This master's current role is: LEADER
I20260806 15:01:33.668459 1339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3ee53d72c6d14537a17757e1ead96d56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee53d72c6d14537a17757e1ead96d56" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40703 } } }
I20260806 15:01:33.668689 1339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [sys.catalog]: This master's current role is: LEADER
I20260806 15:01:33.669275 1344 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:01:33.670023 1344 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:01:33.670117 1344 catalog_manager.cc:1259] Loaded cluster ID: 4bab1b69d1be4c5490895f85a7fe86c8
I20260806 15:01:33.670140 1344 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:01:33.671357 1344 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:01:33.671808 1344 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56: Loaded TSK: 0
I20260806 15:01:33.681072 1344 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56: Generated new TSK 1
I20260806 15:01:33.681175 1344 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 15:01:33.739002 1336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:01:33.739280 1336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:01:33.739327 1336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:01:33.740605 1336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:01:33.740689 1336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:01:33.740738 1336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 15:01:33.742370 1336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:46555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:41611
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=36227
--tserver_master_addrs=127.11.8.126:40703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:01:33.742697 1336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:01:33.742972 1336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:01:33.749383 1375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:01:33.749426 1370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:01:33.749615 1336 server_base.cc:1034] running on GCE node
W20260806 15:01:33.749359 1369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:01:33.750188 1336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:01:33.750577 1336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:01:33.751775 1336 hybrid_clock.cc:648] HybridClock initialized: now 1786028493751750 us; error 42 us; skew 500 ppm
Aug 06 15:01:33 dist-test-slave-68xk krb5kdc[456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028493, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:01:33.754721 1336 init.cc:377] Logged in from keytab as kudu/127.11.8.65@KRBTEST.COM (short username kudu)
I20260806 15:01:33.755192 1336 webserver.cc:460] Webserver started at http://127.11.8.65:36227/ using document root <none> and password file <none>
I20260806 15:01:33.755470 1336 fs_manager.cc:359] Metadata directory not provided
I20260806 15:01:33.755568 1336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:01:33.757050 1336 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:01:33.757862 1386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:01:33.758064 1336 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:01:33.758188 1336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "cc682dee81814201b314bd6b53858bf8"
format_stamp: "Formatted at 2026-08-06 15:01:12 on dist-test-slave-68xk"
server_key: "cfbd07e199a839815a67dda70e31f091"
server_key_iv: "21ebc6b703bac7c47c255ff9062d4e62"
server_key_version: "encryptionkey@0"
I20260806 15:01:33.758600 1336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:01:33.770744 1336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:01:33.772946 1336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:01:33.773116 1336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:01:33.773360 1336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:01:33.773824 1399 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 15:01:33.775897 1336 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 15:01:33.775979 1336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 15:01:33.776026 1336 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 15:01:33.776713 1336 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 15:01:33.776774 1336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 15:01:33.776847 1399 tablet_bootstrap.cc:492] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8: Bootstrap starting.
I20260806 15:01:33.784312 1336 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:41611
I20260806 15:01:33.784477 1551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:41611 every 8 connection(s)
I20260806 15:01:33.784759 1336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
I20260806 15:01:33.786101 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 1336
I20260806 15:01:33.786453 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:45941
--local_ip_for_outbound_sockets=127.11.8.66
--tserver_master_addrs=127.11.8.126:40703
--webserver_port=39133
--webserver_interface=127.11.8.66
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:46555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:01:33.788571 1399 log.cc:824] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8: Log is configured to *not* fsync() on all Append() calls
I20260806 15:01:33.791030 1399 tablet_bootstrap.cc:492] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 15:01:33.791340 1399 tablet_bootstrap.cc:492] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8: Bootstrap complete.
I20260806 15:01:33.792645 1399 ts_tablet_manager.cc:1397] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.005s
I20260806 15:01:33.794886 1399 raft_consensus.cc:348] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 } } peers { permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 } } peers { permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 } }
I20260806 15:01:33.795274 1399 raft_consensus.cc:729] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc682dee81814201b314bd6b53858bf8, State: Initialized, Role: FOLLOWER
Aug 06 15:01:33 dist-test-slave-68xk krb5kdc[456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028493, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:01:33.795451 1399 consensus_queue.cc:260] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 } } peers { permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 } } peers { permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 } }
I20260806 15:01:33.795768 1399 ts_tablet_manager.cc:1428] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
I20260806 15:01:33.800765 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:33.785542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.65:50995)
0806 15:01:33.785767 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:33.785818 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 15:01:33.788823 (+ 3005us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:01:33.789500 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:01:33.789503 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:01:33.789761 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:01:33.789946 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:33.789950 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:33.791261 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:33.791262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:33.792623 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:33.792627 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:33.793342 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:33.793344 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:33.793499 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:33.794154 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:01:33.794164 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:01:33.796087 (+ 1923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:01:33.798709 (+ 2622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:01:33.798718 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:01:33.798720 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:01:33.799026 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:01:33.799381 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:01:33.799382 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:01:33.799382 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:01:33.799531 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:01:33.800057 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:01:33.800067 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:01:33.800250 (+ 183us) client_negotiation.cc:769] Sending connection context
0806 15:01:33.800370 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 15:01:33.800567 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":111,"threads_started":1}
I20260806 15:01:33.800890 1557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:33.785663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:50995 (local address 127.11.8.126:40703)
0806 15:01:33.785893 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0806 15:01:33.785901 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 15:01:33.785945 (+ 44us) server_negotiation.cc:407] Connection header received
0806 15:01:33.789092 (+ 3147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:01:33.789100 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:01:33.789166 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:01:33.789293 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:01:33.790094 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:33.791111 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:33.792793 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:33.793208 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:33.793339 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:33.796242 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:01:33.796247 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:01:33.796248 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:01:33.796263 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:01:33.798607 (+ 2344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:01:33.799131 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:01:33.799136 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:01:33.799136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:01:33.799285 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:01:33.799626 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:01:33.799627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:01:33.799628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:01:33.799952 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:01:33.800054 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 15:01:33.800464 (+ 410us) server_negotiation.cc:299] Negotiation successful
0806 15:01:33.800639 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":85,"threads_started":1}
I20260806 15:01:33.802017 1552 heartbeater.cc:347] Connected to a master server at 127.11.8.126:40703
I20260806 15:01:33.802124 1552 heartbeater.cc:464] Registering TS with master...
I20260806 15:01:33.802304 1552 heartbeater.cc:511] Master 127.11.8.126:40703 requested a full tablet report, sending...
I20260806 15:01:33.803201 1283 ts_manager.cc:194] Registered new tserver with Master: cc682dee81814201b314bd6b53858bf8 (127.11.8.65:41611)
I20260806 15:01:33.804538 1283 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.65@KRBTEST.COM'} at 127.11.8.65:50995
I20260806 15:01:33.805547 1552 heartbeater.cc:503] Master 127.11.8.126:40703 was elected leader, sending a full tablet report...
I20260806 15:01:33.810981 1564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:33.803983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:41611 (local address 127.0.0.1:57312)
0806 15:01:33.806560 (+ 2577us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:33.806600 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 15:01:33.806666 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:01:33.806939 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:01:33.806942 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:01:33.806944 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:01:33.807006 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:33.807007 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:33.808262 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:33.808263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:33.809707 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:33.809710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:33.810701 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:33.810702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:33.810787 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:33.810802 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 15:01:33.810857 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 15:01:33.810905 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2543,"thread_start_us":100,"threads_started":1}
I20260806 15:01:33.811547 1565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:33.804243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57312 (local address 127.11.8.65:41611)
0806 15:01:33.806791 (+ 2548us) server_negotiation.cc:206] Beginning negotiation
0806 15:01:33.806795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:01:33.806804 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:01:33.806836 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:01:33.806838 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:01:33.806852 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:01:33.807068 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:01:33.807110 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:33.808161 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:33.809808 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:33.810592 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:33.811351 (+ 759us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:33.811415 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 15:01:33.811441 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 15:01:33.811478 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2516,"thread_start_us":84,"threads_started":1}
I20260806 15:01:33.812065 1459 tablet_service.cc:1514] Processing DeleteTablet for tablet f681deaef4e44698b2e220d0e1df374c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:01:13 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:57312
I20260806 15:01:33.812717 1567 tablet_replica.cc:330] stopping tablet replica
I20260806 15:01:33.812827 1567 raft_consensus.cc:2229] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:01:33.812894 1567 raft_consensus.cc:2258] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:01:33.813246 1567 ts_tablet_manager.cc:1905] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:01:33.816090 1567 ts_tablet_manager.cc:1918] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:01:33.816171 1567 log.cc:1196] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/wal/wals/f681deaef4e44698b2e220d0e1df374c
I20260806 15:01:33.816401 1567 ts_tablet_manager.cc:1939] T f681deaef4e44698b2e220d0e1df374c P cc682dee81814201b314bd6b53858bf8: Deleting consensus metadata
I20260806 15:01:33.816944 1267 catalog_manager.cc:4841] TS cc682dee81814201b314bd6b53858bf8 (127.11.8.65:41611): tablet f681deaef4e44698b2e220d0e1df374c (table test-table [id=f30ee5b738ca43ff89fdc0faf49a0042]) successfully deleted
W20260806 15:01:33.880229 1560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:01:33.880453 1560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:01:33.880479 1560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:01:33.881723 1560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:01:33.881776 1560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:01:33.881805 1560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 15:01:33.883394 1560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:46555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:45941
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=39133
--tserver_master_addrs=127.11.8.126:40703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:01:33.883674 1560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:01:33.883955 1560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:01:33.889824 1575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:01:33.889819 1582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:01:33.889819 1577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:01:33.889886 1560 server_base.cc:1034] running on GCE node
I20260806 15:01:33.890291 1560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:01:33.890571 1560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:01:33.891752 1560 hybrid_clock.cc:648] HybridClock initialized: now 1786028493891730 us; error 44 us; skew 500 ppm
Aug 06 15:01:33 dist-test-slave-68xk krb5kdc[456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028493, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:01:33.894580 1560 init.cc:377] Logged in from keytab as kudu/127.11.8.66@KRBTEST.COM (short username kudu)
I20260806 15:01:33.895144 1560 webserver.cc:460] Webserver started at http://127.11.8.66:39133/ using document root <none> and password file <none>
I20260806 15:01:33.895442 1560 fs_manager.cc:359] Metadata directory not provided
I20260806 15:01:33.895552 1560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:01:33.896986 1560 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:01:33.897884 1595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:01:33.898085 1560 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:01:33.898205 1560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "5238714247d84a09bbc6bf5d5697baa0"
format_stamp: "Formatted at 2026-08-06 15:01:13 on dist-test-slave-68xk"
server_key: "148c5a7019382d3f584a0e9249abe8a5"
server_key_iv: "b01e7045ad700035fc477a6f6ff21e75"
server_key_version: "encryptionkey@0"
I20260806 15:01:33.898643 1560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:01:33.907564 1560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:01:33.909474 1560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:01:33.909643 1560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:01:33.909880 1560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:01:33.910209 1560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:01:33.910269 1560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:01:33.910305 1560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:01:33.910332 1560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:01:33.916710 1560 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:45941
I20260806 15:01:33.916764 1752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:45941 every 8 connection(s)
I20260806 15:01:33.917132 1560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
Aug 06 15:01:33 dist-test-slave-68xk krb5kdc[456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028493, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:01:33.924489 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 1560
I20260806 15:01:33.924819 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:41739
--local_ip_for_outbound_sockets=127.11.8.67
--tserver_master_addrs=127.11.8.126:40703
--webserver_port=32795
--webserver_interface=127.11.8.67
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:46555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:01:33.927508 1557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:33.917953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:48145 (local address 127.11.8.126:40703)
0806 15:01:33.918025 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 15:01:33.918027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:01:33.918280 (+ 253us) server_negotiation.cc:407] Connection header received
0806 15:01:33.918817 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:01:33.918818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:01:33.918832 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:01:33.918891 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:01:33.919862 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:33.920390 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:33.921444 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:33.921645 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:33.921717 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:33.924097 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:01:33.924098 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:01:33.924098 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:01:33.924104 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:01:33.925769 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:01:33.926325 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:01:33.926326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:01:33.926327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:01:33.926380 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:01:33.926754 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:01:33.926755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:01:33.926756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:01:33.926914 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:01:33.926993 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 15:01:33.927285 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 15:01:33.927388 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 15:01:33.927533 1759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:33.917930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.66:48145)
0806 15:01:33.918128 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:33.918174 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 15:01:33.918725 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:01:33.918948 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:01:33.918952 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:01:33.919574 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:01:33.919781 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:33.919785 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:33.920485 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:33.920486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:33.921361 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:33.921364 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:33.921740 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:33.921741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:33.921857 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:33.922479 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:01:33.922490 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:01:33.924007 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:01:33.925858 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:01:33.925866 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:01:33.925868 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:01:33.926234 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:01:33.926478 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:01:33.926479 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:01:33.926480 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:01:33.926669 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:01:33.927000 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:01:33.927006 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:01:33.927139 (+ 133us) client_negotiation.cc:769] Sending connection context
0806 15:01:33.927240 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 15:01:33.927367 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":119,"threads_started":1}
I20260806 15:01:33.928176 1753 heartbeater.cc:347] Connected to a master server at 127.11.8.126:40703
I20260806 15:01:33.928279 1753 heartbeater.cc:464] Registering TS with master...
I20260806 15:01:33.928488 1753 heartbeater.cc:511] Master 127.11.8.126:40703 requested a full tablet report, sending...
I20260806 15:01:33.928949 1283 ts_manager.cc:194] Registered new tserver with Master: 5238714247d84a09bbc6bf5d5697baa0 (127.11.8.66:45941)
I20260806 15:01:33.929481 1283 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.66@KRBTEST.COM'} at 127.11.8.66:48145
W20260806 15:01:34.011714 1761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:01:34.011920 1761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:01:34.011945 1761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:01:34.013051 1761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:01:34.013088 1761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:01:34.013114 1761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 15:01:34.014585 1761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:46555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:41739
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=32795
--tserver_master_addrs=127.11.8.126:40703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:01:34.014779 1761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:01:34.014953 1761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:01:34.020045 1767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:01:34.020017 1768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:01:34.020363 1761 server_base.cc:1034] running on GCE node
W20260806 15:01:34.020026 1770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:01:34.020637 1761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:01:34.020901 1761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:01:34.022042 1761 hybrid_clock.cc:648] HybridClock initialized: now 1786028494022031 us; error 33 us; skew 500 ppm
Aug 06 15:01:34 dist-test-slave-68xk krb5kdc[456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028494, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:01:34.024675 1761 init.cc:377] Logged in from keytab as kudu/127.11.8.67@KRBTEST.COM (short username kudu)
I20260806 15:01:34.025076 1761 webserver.cc:460] Webserver started at http://127.11.8.67:32795/ using document root <none> and password file <none>
I20260806 15:01:34.025259 1761 fs_manager.cc:359] Metadata directory not provided
I20260806 15:01:34.025308 1761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:01:34.026706 1761 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:01:34.027386 1779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:01:34.027590 1761 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 15:01:34.027675 1761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "28bc371cef284ca4986b189d590a6d7f"
format_stamp: "Formatted at 2026-08-06 15:01:13 on dist-test-slave-68xk"
server_key: "6a6b1956ee51862f4c712fb7150b77fd"
server_key_iv: "8b79200cb7a8a88cfa3cccd708196941"
server_key_version: "encryptionkey@0"
I20260806 15:01:34.027984 1761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:01:34.037755 1761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:01:34.039886 1761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:01:34.040004 1761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:01:34.040216 1761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:01:34.040493 1761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:01:34.040524 1761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:01:34.040546 1761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:01:34.040558 1761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:01:34.048759 1761 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:41739
I20260806 15:01:34.048796 1948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:41739 every 8 connection(s)
I20260806 15:01:34.049279 1761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
I20260806 15:01:34.052305 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 1761
Aug 06 15:01:34 dist-test-slave-68xk krb5kdc[456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028494, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:01:34.059900 1952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:34.052723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41816 (local address 127.11.8.126:40703)
0806 15:01:34.052871 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 15:01:34.052873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:01:34.053055 (+ 182us) server_negotiation.cc:407] Connection header received
0806 15:01:34.053082 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:01:34.053083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:01:34.053095 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:01:34.053164 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:01:34.053770 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:34.054413 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:34.055153 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:34.055445 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:34.055520 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:34.056914 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:01:34.056915 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:01:34.056916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:01:34.056923 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:01:34.058504 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:01:34.058930 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:01:34.058931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:01:34.058932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:01:34.059003 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:01:34.059269 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:01:34.059271 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:01:34.059271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:01:34.059412 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:01:34.059635 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0806 15:01:34.059671 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 15:01:34.059791 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":93,"threads_started":1}
I20260806 15:01:34.060657 1557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:34.050219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:48689 (local address 127.11.8.126:40703)
0806 15:01:34.050294 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 15:01:34.050296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:01:34.050573 (+ 277us) server_negotiation.cc:407] Connection header received
0806 15:01:34.051059 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:01:34.051061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:01:34.051075 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:01:34.051138 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:01:34.052142 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:34.052877 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:34.054129 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:34.054321 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:34.054374 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:34.057279 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:01:34.057281 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:01:34.057281 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:01:34.057289 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:01:34.059016 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:01:34.059581 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:01:34.059583 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:01:34.059583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:01:34.059642 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:01:34.059922 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:01:34.059923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:01:34.059924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:01:34.060054 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:01:34.060118 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 15:01:34.060471 (+ 353us) server_negotiation.cc:299] Negotiation successful
0806 15:01:34.060576 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 15:01:34.061004 1951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:34.050224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.67:48689)
0806 15:01:34.050403 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:34.050488 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0806 15:01:34.050961 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:01:34.051240 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:01:34.051244 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:01:34.051818 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:01:34.052027 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:34.052032 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:34.052991 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:34.052992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:34.054028 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:34.054032 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:34.054407 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:34.054408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:34.054586 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:34.055163 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:01:34.055173 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:01:34.057180 (+ 2007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:01:34.059150 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:01:34.059157 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:01:34.059159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:01:34.059502 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:01:34.059720 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:01:34.059721 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:01:34.059721 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:01:34.059856 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:01:34.060149 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:01:34.060156 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:01:34.060327 (+ 171us) client_negotiation.cc:769] Sending connection context
0806 15:01:34.060436 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 15:01:34.060601 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":94,"threads_started":1}
I20260806 15:01:34.061492 1949 heartbeater.cc:347] Connected to a master server at 127.11.8.126:40703
I20260806 15:01:34.061626 1949 heartbeater.cc:464] Registering TS with master...
I20260806 15:01:34.061854 1949 heartbeater.cc:511] Master 127.11.8.126:40703 requested a full tablet report, sending...
I20260806 15:01:34.062295 1283 ts_manager.cc:194] Registered new tserver with Master: 28bc371cef284ca4986b189d590a6d7f (127.11.8.67:41739)
I20260806 15:01:34.062712 1283 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.67@KRBTEST.COM'} at 127.11.8.67:48689
I20260806 15:01:34.071854 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:01:34.074622 1557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:34.072309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41824 (local address 127.11.8.126:40703)
0806 15:01:34.072385 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 15:01:34.072387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:01:34.072451 (+ 64us) server_negotiation.cc:407] Connection header received
0806 15:01:34.072507 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:01:34.072508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:01:34.072543 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:01:34.072603 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:01:34.072699 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:34.073157 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:34.073970 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:34.074143 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:34.074193 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:34.074357 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:01:34.074399 (+ 42us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 15:01:34.074577 (+ 178us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41824: token expired
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 15:01:34.074710 1557 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41824: token expired
W20260806 15:01:34.074891 1180 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.11.8.126:40703)
I20260806 15:01:34.080888 1557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:34.075159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41840 (local address 127.11.8.126:40703)
0806 15:01:34.075272 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 15:01:34.075274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:01:34.075281 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:01:34.075299 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:01:34.075299 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:01:34.075306 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:01:34.075354 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:01:34.075912 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:34.076346 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:34.077103 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:34.077289 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:34.077340 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:34.078653 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:01:34.078655 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:01:34.078655 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:01:34.078661 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:01:34.079916 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:01:34.080266 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:01:34.080267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:01:34.080267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:01:34.080317 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:01:34.080496 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:01:34.080497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:01:34.080497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:01:34.080613 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:01:34.080688 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 15:01:34.080781 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 15:01:34.080833 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260806 15:01:34.082168 1283 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41840:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:01:34.083926 1283 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:01:34.086261 1459 tablet_service.cc:1467] Processing CreateTablet for tablet 1d830705329f4e17854c0113c51b2e00 (DEFAULT_TABLE table=test-table [id=cad1cc781668488cae29a041c32d7e5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:01:34.086503 1459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d830705329f4e17854c0113c51b2e00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:01:34.087971 1399 tablet_bootstrap.cc:492] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8: Bootstrap starting.
I20260806 15:01:34.088326 1399 tablet_bootstrap.cc:654] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8: Neither blocks nor log segments found. Creating new log.
I20260806 15:01:34.089716 1399 tablet_bootstrap.cc:492] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8: No bootstrap required, opened a new log
I20260806 15:01:34.089735 1564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:34.085908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:41739 (local address 127.0.0.1:35556)
0806 15:01:34.086285 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:34.086297 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:01:34.086341 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:01:34.086655 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:01:34.086656 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:01:34.086658 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:01:34.086702 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:34.086703 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:34.087648 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:34.087650 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:34.088675 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:34.088678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:34.089479 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:34.089479 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:34.089529 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:34.089532 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:01:34.089607 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 15:01:34.089653 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356}
I20260806 15:01:34.089749 1956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:34.086039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35556 (local address 127.11.8.67:41739)
0806 15:01:34.086483 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0806 15:01:34.086488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:01:34.086503 (+ 15us) server_negotiation.cc:407] Connection header received
0806 15:01:34.086538 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:01:34.086558 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:01:34.086576 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:01:34.086658 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:01:34.086752 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:34.087564 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:34.088771 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:34.089396 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:34.089473 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:34.089532 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 15:01:34.089630 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 15:01:34.089669 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":162,"threads_started":1}
I20260806 15:01:34.089767 1399 ts_tablet_manager.cc:1397] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260806 15:01:34.089874 1399 raft_consensus.cc:348] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 } } peers { permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 } } peers { permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 } }
I20260806 15:01:34.089933 1399 raft_consensus.cc:374] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:01:34.089975 1399 raft_consensus.cc:729] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc682dee81814201b314bd6b53858bf8, State: Initialized, Role: FOLLOWER
I20260806 15:01:34.090020 1399 consensus_queue.cc:260] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 } } peers { permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 } } peers { permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 } }
I20260806 15:01:34.090098 1399 ts_tablet_manager.cc:1428] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 15:01:34.090847 1855 tablet_service.cc:1467] Processing CreateTablet for tablet 1d830705329f4e17854c0113c51b2e00 (DEFAULT_TABLE table=test-table [id=cad1cc781668488cae29a041c32d7e5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:01:34.091199 1855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d830705329f4e17854c0113c51b2e00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:01:34.091508 1957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:34.086031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:45941 (local address 127.0.0.1:41772)
0806 15:01:34.087720 (+ 1689us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:34.087730 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:01:34.087781 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:01:34.088081 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:01:34.088082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:01:34.088084 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:01:34.088125 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:34.088127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:34.089279 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:34.089280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:34.090323 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:34.090326 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:34.091266 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:34.091267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:34.091333 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:34.091337 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:01:34.091407 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 15:01:34.091443 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1669,"thread_start_us":183,"threads_started":1}
I20260806 15:01:34.091521 1958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:34.086169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41772 (local address 127.11.8.66:45941)
0806 15:01:34.086369 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 15:01:34.086374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:01:34.087813 (+ 1439us) server_negotiation.cc:407] Connection header received
0806 15:01:34.087869 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:01:34.087871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:01:34.087888 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:01:34.087974 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:01:34.088212 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:34.089191 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:34.090403 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:34.091178 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:34.091240 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:34.091307 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 15:01:34.091425 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 15:01:34.091458 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":106,"threads_started":1}
I20260806 15:01:34.092178 1661 tablet_service.cc:1467] Processing CreateTablet for tablet 1d830705329f4e17854c0113c51b2e00 (DEFAULT_TABLE table=test-table [id=cad1cc781668488cae29a041c32d7e5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:01:34.092499 1661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d830705329f4e17854c0113c51b2e00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:01:34.094293 1963 tablet_bootstrap.cc:492] T 1d830705329f4e17854c0113c51b2e00 P 28bc371cef284ca4986b189d590a6d7f: Bootstrap starting.
I20260806 15:01:34.095034 1963 tablet_bootstrap.cc:654] T 1d830705329f4e17854c0113c51b2e00 P 28bc371cef284ca4986b189d590a6d7f: Neither blocks nor log segments found. Creating new log.
I20260806 15:01:34.095584 1964 tablet_bootstrap.cc:492] T 1d830705329f4e17854c0113c51b2e00 P 5238714247d84a09bbc6bf5d5697baa0: Bootstrap starting.
I20260806 15:01:34.095706 1963 log.cc:824] T 1d830705329f4e17854c0113c51b2e00 P 28bc371cef284ca4986b189d590a6d7f: Log is configured to *not* fsync() on all Append() calls
I20260806 15:01:34.096321 1964 tablet_bootstrap.cc:654] T 1d830705329f4e17854c0113c51b2e00 P 5238714247d84a09bbc6bf5d5697baa0: Neither blocks nor log segments found. Creating new log.
I20260806 15:01:34.096585 1963 tablet_bootstrap.cc:492] T 1d830705329f4e17854c0113c51b2e00 P 28bc371cef284ca4986b189d590a6d7f: No bootstrap required, opened a new log
I20260806 15:01:34.096643 1963 ts_tablet_manager.cc:1397] T 1d830705329f4e17854c0113c51b2e00 P 28bc371cef284ca4986b189d590a6d7f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:01:34.096901 1964 log.cc:824] T 1d830705329f4e17854c0113c51b2e00 P 5238714247d84a09bbc6bf5d5697baa0: Log is configured to *not* fsync() on all Append() calls
I20260806 15:01:34.097668 1964 tablet_bootstrap.cc:492] T 1d830705329f4e17854c0113c51b2e00 P 5238714247d84a09bbc6bf5d5697baa0: No bootstrap required, opened a new log
I20260806 15:01:34.097767 1964 ts_tablet_manager.cc:1397] T 1d830705329f4e17854c0113c51b2e00 P 5238714247d84a09bbc6bf5d5697baa0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:01:34.098344 1963 raft_consensus.cc:348] T 1d830705329f4e17854c0113c51b2e00 P 28bc371cef284ca4986b189d590a6d7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 } } peers { permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 } } peers { permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 } }
I20260806 15:01:34.098554 1963 raft_consensus.cc:374] T 1d830705329f4e17854c0113c51b2e00 P 28bc371cef284ca4986b189d590a6d7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:01:34.098583 1963 raft_consensus.cc:729] T 1d830705329f4e17854c0113c51b2e00 P 28bc371cef284ca4986b189d590a6d7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28bc371cef284ca4986b189d590a6d7f, State: Initialized, Role: FOLLOWER
I20260806 15:01:34.098708 1963 consensus_queue.cc:260] T 1d830705329f4e17854c0113c51b2e00 P 28bc371cef284ca4986b189d590a6d7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 } } peers { permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 } } peers { permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 } }
I20260806 15:01:34.099004 1963 ts_tablet_manager.cc:1428] T 1d830705329f4e17854c0113c51b2e00 P 28bc371cef284ca4986b189d590a6d7f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 15:01:34.099017 1949 heartbeater.cc:503] Master 127.11.8.126:40703 was elected leader, sending a full tablet report...
I20260806 15:01:34.099706 1964 raft_consensus.cc:348] T 1d830705329f4e17854c0113c51b2e00 P 5238714247d84a09bbc6bf5d5697baa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 } } peers { permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 } } peers { permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 } }
I20260806 15:01:34.099900 1964 raft_consensus.cc:374] T 1d830705329f4e17854c0113c51b2e00 P 5238714247d84a09bbc6bf5d5697baa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:01:34.099952 1964 raft_consensus.cc:729] T 1d830705329f4e17854c0113c51b2e00 P 5238714247d84a09bbc6bf5d5697baa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5238714247d84a09bbc6bf5d5697baa0, State: Initialized, Role: FOLLOWER
I20260806 15:01:34.100070 1964 consensus_queue.cc:260] T 1d830705329f4e17854c0113c51b2e00 P 5238714247d84a09bbc6bf5d5697baa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 } } peers { permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 } } peers { permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 } }
I20260806 15:01:34.100371 1964 ts_tablet_manager.cc:1428] T 1d830705329f4e17854c0113c51b2e00 P 5238714247d84a09bbc6bf5d5697baa0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 15:01:34.100371 1753 heartbeater.cc:503] Master 127.11.8.126:40703 was elected leader, sending a full tablet report...
W20260806 15:01:34.167755 1754 tablet.cc:2369] T 1d830705329f4e17854c0113c51b2e00 P 5238714247d84a09bbc6bf5d5697baa0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:01:34.192512 1562 raft_consensus.cc:482] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:01:34.192660 1562 raft_consensus.cc:504] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 } } peers { permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 } } peers { permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 } }
I20260806 15:01:34.193135 1562 leader_election.cc:288] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28bc371cef284ca4986b189d590a6d7f (127.11.8.67:41739), 5238714247d84a09bbc6bf5d5697baa0 (127.11.8.66:45941)
I20260806 15:01:34.196641 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:34.193336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:41739 (local address 127.11.8.65:36239)
0806 15:01:34.193402 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:34.193416 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:01:34.193465 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:01:34.193725 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:01:34.193726 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:01:34.193728 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:01:34.193793 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:34.193795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:34.194739 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:34.194740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:34.195633 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:34.195636 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:34.196384 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:34.196385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:34.196454 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:34.196459 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 15:01:34.196521 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 15:01:34.196562 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 15:01:34.196704 1956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:34.193461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:36239 (local address 127.11.8.67:41739)
0806 15:01:34.193519 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 15:01:34.193522 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:01:34.193533 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:01:34.193565 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:01:34.193566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:01:34.193580 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:01:34.193681 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:01:34.193885 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:34.194644 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:34.195744 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:34.196299 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:34.196354 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:34.196363 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 15:01:34.196569 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 15:01:34.196614 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 15:01:34.197156 1971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:34.193478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:45941 (local address 127.11.8.65:45369)
0806 15:01:34.193654 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:34.193666 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:01:34.193729 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:01:34.193879 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:01:34.193880 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:01:34.193881 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:01:34.194293 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:34.194296 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:34.195103 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:34.195104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:34.196027 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:34.196030 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:34.196912 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:34.196913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:34.197000 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:34.197004 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:01:34.197058 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 15:01:34.197097 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":107,"threads_started":1}
I20260806 15:01:34.197122 1883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d830705329f4e17854c0113c51b2e00" candidate_uuid: "cc682dee81814201b314bd6b53858bf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc371cef284ca4986b189d590a6d7f" is_pre_election: true
I20260806 15:01:34.197193 1958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:34.193590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:45369 (local address 127.11.8.66:45941)
0806 15:01:34.193638 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 15:01:34.193640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:01:34.193736 (+ 96us) server_negotiation.cc:407] Connection header received
0806 15:01:34.193792 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:01:34.193793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:01:34.193807 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:01:34.193860 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:01:34.194373 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:34.194993 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:34.196125 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:34.196627 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:34.196695 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:34.196705 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 15:01:34.197087 (+ 382us) server_negotiation.cc:299] Negotiation successful
0806 15:01:34.197135 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 15:01:34.197297 1883 raft_consensus.cc:2454] T 1d830705329f4e17854c0113c51b2e00 P 28bc371cef284ca4986b189d590a6d7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc682dee81814201b314bd6b53858bf8 in term 0.
I20260806 15:01:34.197582 1391 leader_election.cc:302] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc371cef284ca4986b189d590a6d7f, cc682dee81814201b314bd6b53858bf8; no voters:
I20260806 15:01:34.197724 1562 raft_consensus.cc:2781] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:01:34.197789 1562 raft_consensus.cc:482] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:01:34.197808 1562 raft_consensus.cc:3037] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:01:34.197742 1684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d830705329f4e17854c0113c51b2e00" candidate_uuid: "cc682dee81814201b314bd6b53858bf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5238714247d84a09bbc6bf5d5697baa0" is_pre_election: true
I20260806 15:01:34.197932 1684 raft_consensus.cc:2454] T 1d830705329f4e17854c0113c51b2e00 P 5238714247d84a09bbc6bf5d5697baa0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc682dee81814201b314bd6b53858bf8 in term 0.
I20260806 15:01:34.198745 1562 raft_consensus.cc:504] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 } } peers { permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 } } peers { permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 } }
I20260806 15:01:34.198890 1562 leader_election.cc:288] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [CANDIDATE]: Term 1 election: Requested vote from peers 28bc371cef284ca4986b189d590a6d7f (127.11.8.67:41739), 5238714247d84a09bbc6bf5d5697baa0 (127.11.8.66:45941)
I20260806 15:01:34.199133 1684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d830705329f4e17854c0113c51b2e00" candidate_uuid: "cc682dee81814201b314bd6b53858bf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5238714247d84a09bbc6bf5d5697baa0"
I20260806 15:01:34.199126 1883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d830705329f4e17854c0113c51b2e00" candidate_uuid: "cc682dee81814201b314bd6b53858bf8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28bc371cef284ca4986b189d590a6d7f"
I20260806 15:01:34.199218 1684 raft_consensus.cc:3037] T 1d830705329f4e17854c0113c51b2e00 P 5238714247d84a09bbc6bf5d5697baa0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:01:34.199216 1883 raft_consensus.cc:3037] T 1d830705329f4e17854c0113c51b2e00 P 28bc371cef284ca4986b189d590a6d7f [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:01:34.200101 1883 raft_consensus.cc:2454] T 1d830705329f4e17854c0113c51b2e00 P 28bc371cef284ca4986b189d590a6d7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc682dee81814201b314bd6b53858bf8 in term 1.
I20260806 15:01:34.200101 1684 raft_consensus.cc:2454] T 1d830705329f4e17854c0113c51b2e00 P 5238714247d84a09bbc6bf5d5697baa0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc682dee81814201b314bd6b53858bf8 in term 1.
I20260806 15:01:34.200340 1391 leader_election.cc:302] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28bc371cef284ca4986b189d590a6d7f, cc682dee81814201b314bd6b53858bf8; no voters:
I20260806 15:01:34.200448 1562 raft_consensus.cc:2781] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:01:34.200592 1562 raft_consensus.cc:686] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [term 1 LEADER]: Becoming Leader. State: Replica: cc682dee81814201b314bd6b53858bf8, State: Running, Role: LEADER
I20260806 15:01:34.200660 1562 consensus_queue.cc:237] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 } } peers { permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 } } peers { permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 } }
I20260806 15:01:34.201566 1283 catalog_manager.cc:5495] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc682dee81814201b314bd6b53858bf8 (127.11.8.65). New cstate: current_term: 1 leader_uuid: "cc682dee81814201b314bd6b53858bf8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 } health_report { overall_health: HEALTHY } } }
I20260806 15:01:34.236420 1565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:34.233971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57326 (local address 127.11.8.65:41611)
0806 15:01:34.234070 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 15:01:34.234072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:01:34.234083 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:01:34.234126 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:01:34.234127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:01:34.234150 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:01:34.234226 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:01:34.234466 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:34.234945 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:34.235709 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:34.235890 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:34.235944 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:34.236108 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:01:34.236210 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:01:34.236266 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 15:01:34.236338 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 15:01:34.236360 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 15:01:34.238173 1883 raft_consensus.cc:1261] T 1d830705329f4e17854c0113c51b2e00 P 28bc371cef284ca4986b189d590a6d7f [term 1 FOLLOWER]: Refusing update from remote peer cc682dee81814201b314bd6b53858bf8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:01:34.238173 1684 raft_consensus.cc:1261] T 1d830705329f4e17854c0113c51b2e00 P 5238714247d84a09bbc6bf5d5697baa0 [term 1 FOLLOWER]: Refusing update from remote peer cc682dee81814201b314bd6b53858bf8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:01:34.238643 1562 consensus_queue.cc:1035] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:01:34.238823 1972 consensus_queue.cc:1035] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:01:34.242297 1981 mvcc.cc:205] Tried to move back new op lower bound from 7315572712396566528 to 7315572712246472704. Current Snapshot: MvccSnapshot[applied={T|T < 7315572712396566528}]
I20260806 15:01:34.244220 1282 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:01:34.244331 1282 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:01:34.245214 1282 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56: Sending DeleteTablet for 3 replicas of tablet 1d830705329f4e17854c0113c51b2e00
I20260806 15:01:34.245572 1459 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d830705329f4e17854c0113c51b2e00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:01:34 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:57312
I20260806 15:01:34.245615 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 1336
I20260806 15:01:34.245628 1661 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d830705329f4e17854c0113c51b2e00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:01:34 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:41772
I20260806 15:01:34.245680 1567 tablet_replica.cc:330] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8: stopping tablet replica
I20260806 15:01:34.245746 1567 raft_consensus.cc:2229] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:01:34.245838 1567 raft_consensus.cc:2258] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:01:34.246186 1992 tablet_replica.cc:330] T 1d830705329f4e17854c0113c51b2e00 P 5238714247d84a09bbc6bf5d5697baa0: stopping tablet replica
I20260806 15:01:34.246171 1855 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d830705329f4e17854c0113c51b2e00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:01:34 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:35556
I20260806 15:01:34.246304 1992 raft_consensus.cc:2229] T 1d830705329f4e17854c0113c51b2e00 P 5238714247d84a09bbc6bf5d5697baa0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:01:34.246389 1993 tablet_replica.cc:330] T 1d830705329f4e17854c0113c51b2e00 P 28bc371cef284ca4986b189d590a6d7f: stopping tablet replica
I20260806 15:01:34.246426 1992 raft_consensus.cc:2258] T 1d830705329f4e17854c0113c51b2e00 P 5238714247d84a09bbc6bf5d5697baa0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:01:34.246531 1993 raft_consensus.cc:2229] T 1d830705329f4e17854c0113c51b2e00 P 28bc371cef284ca4986b189d590a6d7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:01:34.246680 1993 raft_consensus.cc:2258] T 1d830705329f4e17854c0113c51b2e00 P 28bc371cef284ca4986b189d590a6d7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:01:34.248391 1992 ts_tablet_manager.cc:1905] T 1d830705329f4e17854c0113c51b2e00 P 5238714247d84a09bbc6bf5d5697baa0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:01:34.250052 1993 ts_tablet_manager.cc:1905] T 1d830705329f4e17854c0113c51b2e00 P 28bc371cef284ca4986b189d590a6d7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:01:34.250701 1992 ts_tablet_manager.cc:1918] T 1d830705329f4e17854c0113c51b2e00 P 5238714247d84a09bbc6bf5d5697baa0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:01:34.250789 1992 log.cc:1196] T 1d830705329f4e17854c0113c51b2e00 P 5238714247d84a09bbc6bf5d5697baa0: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/wal/wals/1d830705329f4e17854c0113c51b2e00
I20260806 15:01:34.251151 1992 ts_tablet_manager.cc:1939] T 1d830705329f4e17854c0113c51b2e00 P 5238714247d84a09bbc6bf5d5697baa0: Deleting consensus metadata
I20260806 15:01:34.251669 1266 catalog_manager.cc:4841] TS 5238714247d84a09bbc6bf5d5697baa0 (127.11.8.66:45941): tablet 1d830705329f4e17854c0113c51b2e00 (table test-table [id=cad1cc781668488cae29a041c32d7e5f]) successfully deleted
I20260806 15:01:34.252065 1993 ts_tablet_manager.cc:1918] T 1d830705329f4e17854c0113c51b2e00 P 28bc371cef284ca4986b189d590a6d7f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:01:34.252133 1993 log.cc:1196] T 1d830705329f4e17854c0113c51b2e00 P 28bc371cef284ca4986b189d590a6d7f: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/wal/wals/1d830705329f4e17854c0113c51b2e00
I20260806 15:01:34.252368 1993 ts_tablet_manager.cc:1939] T 1d830705329f4e17854c0113c51b2e00 P 28bc371cef284ca4986b189d590a6d7f: Deleting consensus metadata
I20260806 15:01:34.252951 1266 catalog_manager.cc:4841] TS 28bc371cef284ca4986b189d590a6d7f (127.11.8.67:41739): tablet 1d830705329f4e17854c0113c51b2e00 (table test-table [id=cad1cc781668488cae29a041c32d7e5f]) successfully deleted
W20260806 15:01:34.258373 1267 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.65:41611 (error 108)
I20260806 15:01:34.258690 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 1560
I20260806 15:01:34.258893 1957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:34.258751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:41611 (local address 127.0.0.1:57342)
0806 15:01:34.258813 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:34.258855 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.65:41611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 15:01:34.260214 1267 catalog_manager.cc:4584] TS cc682dee81814201b314bd6b53858bf8 (127.11.8.65:41611): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1d830705329f4e17854c0113c51b2e00: Network error: Client connection negotiation failed: client connection to 127.11.8.65:41611: connect: Connection refused (error 111)
I20260806 15:01:34.268401 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 1761
I20260806 15:01:34.277896 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 1238
I20260806 15:01:54.289626 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:40703
--webserver_interface=127.11.8.126
--webserver_port=41273
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:46555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:40703
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:01:54.374931 2003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:01:54.375144 2003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:01:54.375167 2003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:01:54.376325 2003 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 15:01:54.376367 2003 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 15:01:54.376381 2003 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 15:01:54.376392 2003 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:01:54.376406 2003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:01:54.376420 2003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:01:54.376471 2003 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:01:54.376488 2003 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:01:54.378000 2003 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:46555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:40703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:40703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=41273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:01:54.378266 2003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:01:54.378549 2003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:01:54.383848 2012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:01:54.383873 2016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:01:54.383873 2011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:01:54.383913 2003 server_base.cc:1034] running on GCE node
I20260806 15:01:54.384447 2003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:01:54.384748 2003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:01:54.385919 2003 hybrid_clock.cc:648] HybridClock initialized: now 1786028514385911 us; error 37 us; skew 500 ppm
Aug 06 15:01:54 dist-test-slave-68xk krb5kdc[456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028514, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:01:54.395021 2003 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 15:01:54.395565 2003 webserver.cc:460] Webserver started at http://127.11.8.126:41273/ using document root <none> and password file <none>
I20260806 15:01:54.395833 2003 fs_manager.cc:359] Metadata directory not provided
I20260806 15:01:54.395910 2003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:01:54.397334 2003 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:01:54.398056 2028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:01:54.398353 2003 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:01:54.398528 2003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "3ee53d72c6d14537a17757e1ead96d56"
format_stamp: "Formatted at 2026-08-06 15:01:12 on dist-test-slave-68xk"
server_key: "28867198b781e661e2450bb4470137da"
server_key_iv: "d9693c96ace7a2708ed265bac6fe7abd"
server_key_version: "encryptionkey@0"
I20260806 15:01:54.398943 2003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:01:54.417160 2003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:01:54.419215 2003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:01:54.419394 2003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:01:54.424058 2003 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:40703
I20260806 15:01:54.424101 2094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:40703 every 8 connection(s)
I20260806 15:01:54.424523 2003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:01:54.425989 2095 sys_catalog.cc:263] Verifying existing consensus state
I20260806 15:01:54.426206 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 2003
I20260806 15:01:54.426570 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:41611
--local_ip_for_outbound_sockets=127.11.8.65
--tserver_master_addrs=127.11.8.126:40703
--webserver_port=36227
--webserver_interface=127.11.8.65
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:46555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:01:54.426904 2095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56: Bootstrap starting.
I20260806 15:01:54.435943 2095 log.cc:824] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56: Log is configured to *not* fsync() on all Append() calls
I20260806 15:01:54.438198 2095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 15:01:54.438583 2095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56: Bootstrap complete.
I20260806 15:01:54.440864 2095 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee53d72c6d14537a17757e1ead96d56" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40703 } }
I20260806 15:01:54.441111 2095 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ee53d72c6d14537a17757e1ead96d56, State: Initialized, Role: FOLLOWER
I20260806 15:01:54.441303 2095 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee53d72c6d14537a17757e1ead96d56" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40703 } }
I20260806 15:01:54.441382 2095 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:01:54.441434 2095 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:01:54.441481 2095 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [term 2 FOLLOWER]: Advancing to term 3
I20260806 15:01:54.442487 2095 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee53d72c6d14537a17757e1ead96d56" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40703 } }
I20260806 15:01:54.442615 2095 leader_election.cc:302] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ee53d72c6d14537a17757e1ead96d56; no voters:
I20260806 15:01:54.442809 2095 leader_election.cc:288] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 15:01:54.442845 2099 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 15:01:54.442965 2099 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [term 3 LEADER]: Becoming Leader. State: Replica: 3ee53d72c6d14537a17757e1ead96d56, State: Running, Role: LEADER
I20260806 15:01:54.443099 2095 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:01:54.443118 2099 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee53d72c6d14537a17757e1ead96d56" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40703 } }
I20260806 15:01:54.443538 2101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "3ee53d72c6d14537a17757e1ead96d56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee53d72c6d14537a17757e1ead96d56" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40703 } } }
I20260806 15:01:54.443655 2101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [sys.catalog]: This master's current role is: LEADER
I20260806 15:01:54.443902 2103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ee53d72c6d14537a17757e1ead96d56. Latest consensus state: current_term: 3 leader_uuid: "3ee53d72c6d14537a17757e1ead96d56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ee53d72c6d14537a17757e1ead96d56" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40703 } } }
I20260806 15:01:54.443979 2108 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:01:54.444135 2103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56 [sys.catalog]: This master's current role is: LEADER
I20260806 15:01:54.445070 2108 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:01:54.445181 2108 catalog_manager.cc:1259] Loaded cluster ID: 4bab1b69d1be4c5490895f85a7fe86c8
I20260806 15:01:54.445274 2108 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:01:54.446204 2108 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:01:54.446405 2108 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56: Loaded TSK: 0
I20260806 15:01:54.446502 2108 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3ee53d72c6d14537a17757e1ead96d56: Loaded TSK: 1
F20260806 15:01:54.446738 2108 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f52c49ba9ad google::LogMessage::Fail() at ??:0
@ 0x7f52c49bd88a google::LogMessage::SendToLog() at ??:0
@ 0x7f52c49ba4de google::LogMessage::Flush() at ??:0
@ 0x7f52c49bba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f52c7127104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f52c7124d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f52cc9141cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f52cc91448e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f52cc914594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f52cc8dac48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f52cc8e4ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f52c68880d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f52c687d954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f52c76e16db start_thread at ??:0
@ 0x7f52c34b861f clone at ??:0
W20260806 15:01:54.530887 2097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:01:54.531116 2097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:01:54.531153 2097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:01:54.532370 2097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:01:54.532434 2097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:01:54.532480 2097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 15:01:54.533980 2097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:46555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:41611
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=36227
--tserver_master_addrs=127.11.8.126:40703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:01:54.534257 2097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:01:54.534543 2097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:01:54.539937 2151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:01:54.539965 2152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:01:54.540071 2156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:01:54.540396 2097 server_base.cc:1034] running on GCE node
I20260806 15:01:54.540649 2097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:01:54.540899 2097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:01:54.542083 2097 hybrid_clock.cc:648] HybridClock initialized: now 1786028514542052 us; error 49 us; skew 500 ppm
Aug 06 15:01:54 dist-test-slave-68xk krb5kdc[456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028514, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:01:54.544976 2097 init.cc:377] Logged in from keytab as kudu/127.11.8.65@KRBTEST.COM (short username kudu)
I20260806 15:01:54.545612 2097 webserver.cc:460] Webserver started at http://127.11.8.65:36227/ using document root <none> and password file <none>
I20260806 15:01:54.545864 2097 fs_manager.cc:359] Metadata directory not provided
I20260806 15:01:54.545939 2097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:01:54.547488 2097 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:01:54.548404 2170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:01:54.548623 2097 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:01:54.548758 2097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "cc682dee81814201b314bd6b53858bf8"
format_stamp: "Formatted at 2026-08-06 15:01:12 on dist-test-slave-68xk"
server_key: "cfbd07e199a839815a67dda70e31f091"
server_key_iv: "21ebc6b703bac7c47c255ff9062d4e62"
server_key_version: "encryptionkey@0"
I20260806 15:01:54.549145 2097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:01:54.567126 2097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:01:54.569669 2097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:01:54.569864 2097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:01:54.570137 2097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:01:54.570879 2179 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 15:01:54.573256 2097 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 15:01:54.573309 2097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260806 15:01:54.573355 2097 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 15:01:54.574355 2097 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 15:01:54.574393 2097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 15:01:54.574574 2179 tablet_bootstrap.cc:492] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8: Bootstrap starting.
I20260806 15:01:54.581879 2097 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:41611
I20260806 15:01:54.582015 2331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:41611 every 8 connection(s)
I20260806 15:01:54.582310 2097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
I20260806 15:01:54.582867 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 2097
I20260806 15:01:54.583151 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:45941
--local_ip_for_outbound_sockets=127.11.8.66
--tserver_master_addrs=127.11.8.126:40703
--webserver_port=39133
--webserver_interface=127.11.8.66
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:46555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:01:54.587391 2179 log.cc:824] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8: Log is configured to *not* fsync() on all Append() calls
I20260806 15:01:54.590823 2179 tablet_bootstrap.cc:492] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 15:01:54.591185 2179 tablet_bootstrap.cc:492] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8: Bootstrap complete.
I20260806 15:01:54.592530 2179 ts_tablet_manager.cc:1397] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.013s
I20260806 15:01:54.594449 2179 raft_consensus.cc:348] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 } } peers { permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 } } peers { permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 } }
I20260806 15:01:54.594864 2179 raft_consensus.cc:729] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc682dee81814201b314bd6b53858bf8, State: Initialized, Role: FOLLOWER
I20260806 15:01:54.595041 2179 consensus_queue.cc:260] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 } } peers { permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 } } peers { permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 } }
I20260806 15:01:54.595367 2179 ts_tablet_manager.cc:1428] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
W20260806 15:01:54.667728 2334 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:01:54.583085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.65:50461)
0806 15:01:54.583498 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:54.583549 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 15:01:54.584291 (+ 742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:01:54.667518 (+ 83227us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":109,"threads_started":1}
W20260806 15:01:54.668005 2332 heartbeater.cc:644] Failed to heartbeat to 127.11.8.126:40703 (0 consecutive failures): Network error: Failed to ping master at 127.11.8.126:40703: Client connection negotiation failed: client connection to 127.11.8.126:40703: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 15:01:54.668418 2334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:54.668301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.65:38195)
0806 15:01:54.668355 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:54.668397 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 15:01:54.668845 2334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:54.668697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.65:46815)
0806 15:01:54.668758 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:54.668785 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 15:01:54.668946 2332 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 15:01:54.672058 2335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:01:54.672267 2335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:01:54.672317 2335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:01:54.673467 2335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:01:54.673527 2335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:01:54.673584 2335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 15:01:54.675097 2335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:46555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:45941
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=39133
--tserver_master_addrs=127.11.8.126:40703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:01:54.675350 2335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:01:54.675582 2335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:01:54.680585 2346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:01:54.680619 2348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:01:54.680915 2335 server_base.cc:1034] running on GCE node
W20260806 15:01:54.680593 2345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:01:54.681176 2335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:01:54.681434 2335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:01:54.682595 2335 hybrid_clock.cc:648] HybridClock initialized: now 1786028514682580 us; error 34 us; skew 500 ppm
Aug 06 15:01:54 dist-test-slave-68xk krb5kdc[456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028514, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:01:54.685178 2335 init.cc:377] Logged in from keytab as kudu/127.11.8.66@KRBTEST.COM (short username kudu)
I20260806 15:01:54.685668 2335 webserver.cc:460] Webserver started at http://127.11.8.66:39133/ using document root <none> and password file <none>
I20260806 15:01:54.685899 2335 fs_manager.cc:359] Metadata directory not provided
I20260806 15:01:54.685971 2335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:01:54.687393 2335 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:01:54.688076 2356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:01:54.688280 2335 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:01:54.688386 2335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "5238714247d84a09bbc6bf5d5697baa0"
format_stamp: "Formatted at 2026-08-06 15:01:13 on dist-test-slave-68xk"
server_key: "148c5a7019382d3f584a0e9249abe8a5"
server_key_iv: "b01e7045ad700035fc477a6f6ff21e75"
server_key_version: "encryptionkey@0"
I20260806 15:01:54.688740 2335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:01:54.726508 2335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:01:54.729048 2335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:01:54.729228 2335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:01:54.729494 2335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:01:54.729835 2335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:01:54.729888 2335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:01:54.729930 2335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:01:54.729970 2335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:01:54.737030 2335 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:45941
I20260806 15:01:54.737082 2513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:45941 every 8 connection(s)
I20260806 15:01:54.737483 2335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
I20260806 15:01:54.738713 2516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:54.738233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.66:35193)
0806 15:01:54.738426 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:54.738539 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
W20260806 15:01:54.739153 2514 heartbeater.cc:644] Failed to heartbeat to 127.11.8.126:40703 (0 consecutive failures): Network error: Failed to ping master at 127.11.8.126:40703: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
I20260806 15:01:54.739457 2516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:54.739353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.66:41083)
0806 15:01:54.739405 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:54.739437 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 15:01:54.739785 2516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:54.739676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.66:37967)
0806 15:01:54.739726 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:54.739755 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 15:01:54.739919 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 2335
W20260806 15:01:54.739925 2514 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 15:01:54.740202 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:41739
--local_ip_for_outbound_sockets=127.11.8.67
--tserver_master_addrs=127.11.8.126:40703
--webserver_port=32795
--webserver_interface=127.11.8.67
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:46555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:01:54.823892 2517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:01:54.824101 2517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:01:54.824126 2517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:01:54.825208 2517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:01:54.825244 2517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:01:54.825269 2517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 15:01:54.826761 2517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:46555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:41739
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=32795
--tserver_master_addrs=127.11.8.126:40703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:01:54.826938 2517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:01:54.827116 2517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:01:54.832195 2526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:01:54.832229 2522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:01:54.832245 2523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:01:54.832268 2517 server_base.cc:1034] running on GCE node
I20260806 15:01:54.832548 2517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:01:54.832810 2517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:01:54.833984 2517 hybrid_clock.cc:648] HybridClock initialized: now 1786028514833967 us; error 46 us; skew 500 ppm
Aug 06 15:01:54 dist-test-slave-68xk krb5kdc[456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028514, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:01:54.836575 2517 init.cc:377] Logged in from keytab as kudu/127.11.8.67@KRBTEST.COM (short username kudu)
I20260806 15:01:54.837054 2517 webserver.cc:460] Webserver started at http://127.11.8.67:32795/ using document root <none> and password file <none>
I20260806 15:01:54.837281 2517 fs_manager.cc:359] Metadata directory not provided
I20260806 15:01:54.837329 2517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:01:54.838713 2517 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:01:54.839331 2536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:01:54.839536 2517 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:01:54.839668 2517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "28bc371cef284ca4986b189d590a6d7f"
format_stamp: "Formatted at 2026-08-06 15:01:13 on dist-test-slave-68xk"
server_key: "6a6b1956ee51862f4c712fb7150b77fd"
server_key_iv: "8b79200cb7a8a88cfa3cccd708196941"
server_key_version: "encryptionkey@0"
I20260806 15:01:54.840022 2517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:01:54.864939 2517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:01:54.867269 2517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:01:54.867467 2517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:01:54.867751 2517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:01:54.868104 2517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:01:54.868162 2517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:01:54.868206 2517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:01:54.868250 2517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:01:54.874502 2517 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:41739
I20260806 15:01:54.874567 2685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:41739 every 8 connection(s)
I20260806 15:01:54.874904 2517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
I20260806 15:01:54.875974 2689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:54.875600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.67:56639)
0806 15:01:54.875759 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:54.875832 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":76,"threads_started":1}
I20260806 15:01:54.876287 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 2517
W20260806 15:01:54.876380 2687 heartbeater.cc:644] Failed to heartbeat to 127.11.8.126:40703 (0 consecutive failures): Network error: Failed to ping master at 127.11.8.126:40703: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
I20260806 15:01:54.876708 2689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:54.876584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.67:45867)
0806 15:01:54.876635 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:54.876663 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 15:01:54.877107 2689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:54.876999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.67:41321)
0806 15:01:54.877050 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:54.877080 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 15:01:54.877211 2687 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 15:01:55.669585 2692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:55.669313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.65:44831)
0806 15:01:55.669469 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:55.669510 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":68,"threads_started":1}
I20260806 15:01:55.740448 2693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:55.740231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.66:45513)
0806 15:01:55.740360 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:55.740393 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":61,"threads_started":1}
I20260806 15:01:55.877784 2694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:55.877547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.67:53973)
0806 15:01:55.877679 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:55.877723 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":67,"threads_started":1}
I20260806 15:01:56.196250 2695 raft_consensus.cc:482] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:01:56.196388 2695 raft_consensus.cc:504] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 } } peers { permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 } } peers { permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 } }
I20260806 15:01:56.196798 2695 leader_election.cc:288] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 28bc371cef284ca4986b189d590a6d7f (127.11.8.67:41739), 5238714247d84a09bbc6bf5d5697baa0 (127.11.8.66:45941)
Aug 06 15:01:56 dist-test-slave-68xk krb5kdc[456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028514, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for kudu/127.11.8.66@KRBTEST.COM
Aug 06 15:01:56 dist-test-slave-68xk krb5kdc[456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028514, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for kudu/127.11.8.67@KRBTEST.COM
I20260806 15:01:56.210141 2696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:56.197046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:41739 (local address 127.11.8.65:55785)
0806 15:01:56.197189 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:56.197202 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:01:56.197270 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:01:56.198507 (+ 1237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:01:56.198510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:01:56.199096 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:01:56.199414 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:56.199423 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:56.201151 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:56.201153 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:56.202685 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:56.202689 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:56.203471 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:56.203472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:56.203616 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:56.204268 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:01:56.204278 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:01:56.206407 (+ 2129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:01:56.208637 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:01:56.208639 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:01:56.208639 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:01:56.208880 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:01:56.209188 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:01:56.209189 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:01:56.209189 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:01:56.209250 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:01:56.209700 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:01:56.209701 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:01:56.209744 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 15:01:56.209811 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 15:01:56.210034 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":91,"thread_start_us":60,"threads_started":1}
I20260806 15:01:56.210206 2700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:56.197414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:32825 (local address 127.11.8.66:45941)
0806 15:01:56.197579 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 15:01:56.197602 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 15:01:56.197633 (+ 31us) server_negotiation.cc:407] Connection header received
0806 15:01:56.198162 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:01:56.198165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:01:56.198217 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:01:56.198385 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:01:56.199575 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:56.201012 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:56.202804 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:56.203425 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:56.203553 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:56.206132 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:01:56.206137 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:01:56.206137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:01:56.206149 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:01:56.208152 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:01:56.208666 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:01:56.208670 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:01:56.208671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:01:56.208788 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:01:56.209105 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:01:56.209106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:01:56.209106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:01:56.209441 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:01:56.209505 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 15:01:56.209932 (+ 427us) server_negotiation.cc:299] Negotiation successful
0806 15:01:56.210061 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
I20260806 15:01:56.210225 2697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:56.197199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:55785 (local address 127.11.8.67:41739)
0806 15:01:56.197431 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 15:01:56.197448 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 15:01:56.197484 (+ 36us) server_negotiation.cc:407] Connection header received
0806 15:01:56.198149 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:01:56.198153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:01:56.198217 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:01:56.198385 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:01:56.199575 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:56.201015 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:56.202798 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:01:56.203363 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:01:56.203454 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:56.206543 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:01:56.206547 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:01:56.206547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:01:56.206568 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:01:56.208494 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:01:56.208978 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:01:56.208981 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:01:56.208981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:01:56.209111 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:01:56.209324 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:01:56.209325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:01:56.209325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:01:56.209612 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:01:56.209695 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 15:01:56.209907 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 15:01:56.210052 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":98,"threads_started":1}
I20260806 15:01:56.210258 2699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:56.197244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:45941 (local address 127.11.8.65:32825)
0806 15:01:56.197360 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:56.197371 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:01:56.197433 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:01:56.198506 (+ 1073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:01:56.198510 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:01:56.199109 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:01:56.199414 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:56.199426 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:56.201154 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:56.201154 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:56.202685 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:01:56.202689 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:01:56.203535 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:01:56.203537 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:01:56.203627 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:01:56.204315 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:01:56.204325 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:01:56.205993 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:01:56.208246 (+ 2253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:01:56.208254 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:01:56.208256 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:01:56.208569 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:01:56.208874 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:01:56.208875 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:01:56.208875 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:01:56.209028 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:01:56.209536 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:01:56.209544 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:01:56.209720 (+ 176us) client_negotiation.cc:769] Sending connection context
0806 15:01:56.209846 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 15:01:56.210123 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":213,"thread_start_us":67,"threads_started":1}
W20260806 15:01:56.211377 2174 leader_election.cc:341] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 28bc371cef284ca4986b189d590a6d7f (127.11.8.67:41739): Not found: Tablet not found: 1d830705329f4e17854c0113c51b2e00
W20260806 15:01:56.211458 2174 leader_election.cc:341] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5238714247d84a09bbc6bf5d5697baa0 (127.11.8.66:45941): Not found: Tablet not found: 1d830705329f4e17854c0113c51b2e00
I20260806 15:01:56.211476 2174 leader_election.cc:302] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc682dee81814201b314bd6b53858bf8; no voters: 28bc371cef284ca4986b189d590a6d7f, 5238714247d84a09bbc6bf5d5697baa0
I20260806 15:01:56.211580 2695 raft_consensus.cc:2726] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 15:01:56.670238 2696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:56.670094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.65:51851)
0806 15:01:56.670162 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:56.670198 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 15:01:56.741227 2701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:56.741012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.66:58313)
0806 15:01:56.741151 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:56.741188 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":64,"threads_started":1}
I20260806 15:01:56.878530 2704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:56.878255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.67:38197)
0806 15:01:56.878386 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:56.878459 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":64,"threads_started":1}
I20260806 15:01:57.670907 2705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:57.670666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.65:40947)
0806 15:01:57.670813 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:57.670853 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":76,"threads_started":1}
I20260806 15:01:57.741943 2707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:57.741681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.66:43247)
0806 15:01:57.741859 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:57.741900 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":86,"threads_started":1}
I20260806 15:01:57.879325 2709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:57.879040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.67:35899)
0806 15:01:57.879217 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:57.879263 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":85,"threads_started":1}
I20260806 15:01:58.030138 2712 raft_consensus.cc:482] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:01:58.030305 2712 raft_consensus.cc:504] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 } } peers { permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 } } peers { permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 } }
I20260806 15:01:58.030511 2712 leader_election.cc:288] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 28bc371cef284ca4986b189d590a6d7f (127.11.8.67:41739), 5238714247d84a09bbc6bf5d5697baa0 (127.11.8.66:45941)
W20260806 15:01:58.030931 2174 leader_election.cc:341] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 28bc371cef284ca4986b189d590a6d7f (127.11.8.67:41739): Not found: Tablet not found: 1d830705329f4e17854c0113c51b2e00
W20260806 15:01:58.031015 2174 leader_election.cc:341] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5238714247d84a09bbc6bf5d5697baa0 (127.11.8.66:45941): Not found: Tablet not found: 1d830705329f4e17854c0113c51b2e00
I20260806 15:01:58.031033 2174 leader_election.cc:302] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc682dee81814201b314bd6b53858bf8; no voters: 28bc371cef284ca4986b189d590a6d7f, 5238714247d84a09bbc6bf5d5697baa0
I20260806 15:01:58.031113 2712 raft_consensus.cc:2726] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 15:01:58.671733 2713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:58.671488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.65:58141)
0806 15:01:58.671637 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:58.671677 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":66,"threads_started":1}
I20260806 15:01:58.742887 2714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:58.742590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.66:57567)
0806 15:01:58.742797 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:58.742845 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":113,"threads_started":1}
I20260806 15:01:58.880172 2715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:58.879956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.67:58585)
0806 15:01:58.880095 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:58.880129 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":67,"threads_started":1}
I20260806 15:01:59.672647 2717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:59.672405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.65:57135)
0806 15:01:59.672558 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:59.672598 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":71,"threads_started":1}
I20260806 15:01:59.743633 2718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:59.743386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.66:45079)
0806 15:01:59.743520 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:59.743560 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":67,"threads_started":1}
I20260806 15:01:59.880918 2719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:01:59.880700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.67:52555)
0806 15:01:59.880833 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 15:01:59.880875 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":80,"threads_started":1}
I20260806 15:02:00.673414 2720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:00.673182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.65:52549)
0806 15:02:00.673323 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:00.673366 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":80,"threads_started":1}
I20260806 15:02:00.716130 2721 raft_consensus.cc:482] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:02:00.716238 2721 raft_consensus.cc:504] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 } } peers { permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 } } peers { permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 } }
I20260806 15:02:00.716426 2721 leader_election.cc:288] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 28bc371cef284ca4986b189d590a6d7f (127.11.8.67:41739), 5238714247d84a09bbc6bf5d5697baa0 (127.11.8.66:45941)
W20260806 15:02:00.716887 2174 leader_election.cc:341] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 28bc371cef284ca4986b189d590a6d7f (127.11.8.67:41739): Not found: Tablet not found: 1d830705329f4e17854c0113c51b2e00
W20260806 15:02:00.716961 2174 leader_election.cc:341] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5238714247d84a09bbc6bf5d5697baa0 (127.11.8.66:45941): Not found: Tablet not found: 1d830705329f4e17854c0113c51b2e00
I20260806 15:02:00.717010 2174 leader_election.cc:302] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc682dee81814201b314bd6b53858bf8; no voters: 28bc371cef284ca4986b189d590a6d7f, 5238714247d84a09bbc6bf5d5697baa0
I20260806 15:02:00.717104 2721 raft_consensus.cc:2726] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 15:02:00.744357 2722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:00.744149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.66:38185)
0806 15:02:00.744281 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:00.744319 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":64,"threads_started":1}
I20260806 15:02:00.881682 2726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:00.881409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.67:52291)
0806 15:02:00.881581 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:00.881625 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":86,"threads_started":1}
I20260806 15:02:01.674311 2728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:01.674015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.65:60529)
0806 15:02:01.674186 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:01.674229 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":111,"threads_started":1}
I20260806 15:02:01.745178 2729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:01.744919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.66:46925)
0806 15:02:01.745086 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:01.745125 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":99,"threads_started":1}
I20260806 15:02:01.882575 2730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:01.882239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.67:44653)
0806 15:02:01.882407 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:01.882517 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":85,"threads_started":1}
I20260806 15:02:02.674978 2731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:02.674745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.65:51667)
0806 15:02:02.674892 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:02.674926 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":85,"threads_started":1}
I20260806 15:02:02.746007 2733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:02.745756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.66:54381)
0806 15:02:02.745921 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:02.745962 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":86,"threads_started":1}
I20260806 15:02:02.883371 2734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:02.883148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.67:38481)
0806 15:02:02.883285 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:02.883327 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":64,"threads_started":1}
I20260806 15:02:03.675760 2735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:03.675518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.65:52977)
0806 15:02:03.675662 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:03.675703 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":65,"threads_started":1}
I20260806 15:02:03.746850 2736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:03.746599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.66:51333)
0806 15:02:03.746763 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:03.746804 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":96,"threads_started":1}
I20260806 15:02:03.884168 2737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:03.883941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.67:34269)
0806 15:02:03.884071 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:03.884115 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":65,"threads_started":1}
I20260806 15:02:04.676476 2739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:04.676251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.65:36811)
0806 15:02:04.676388 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:04.676425 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":61,"threads_started":1}
I20260806 15:02:04.747581 2740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:04.747322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.66:33303)
0806 15:02:04.747491 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:04.747533 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":100,"threads_started":1}
I20260806 15:02:04.885030 2741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:04.884717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.67:39413)
0806 15:02:04.884894 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:04.884950 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":89,"threads_started":1}
I20260806 15:02:05.035101 2742 raft_consensus.cc:482] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:02:05.035257 2742 raft_consensus.cc:504] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28bc371cef284ca4986b189d590a6d7f" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41739 } } peers { permanent_uuid: "5238714247d84a09bbc6bf5d5697baa0" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 45941 } } peers { permanent_uuid: "cc682dee81814201b314bd6b53858bf8" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 41611 } }
I20260806 15:02:05.035487 2742 leader_election.cc:288] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 28bc371cef284ca4986b189d590a6d7f (127.11.8.67:41739), 5238714247d84a09bbc6bf5d5697baa0 (127.11.8.66:45941)
W20260806 15:02:05.035918 2174 leader_election.cc:341] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 28bc371cef284ca4986b189d590a6d7f (127.11.8.67:41739): Not found: Tablet not found: 1d830705329f4e17854c0113c51b2e00
W20260806 15:02:05.036024 2174 leader_election.cc:341] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5238714247d84a09bbc6bf5d5697baa0 (127.11.8.66:45941): Not found: Tablet not found: 1d830705329f4e17854c0113c51b2e00
I20260806 15:02:05.036083 2174 leader_election.cc:302] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc682dee81814201b314bd6b53858bf8; no voters: 28bc371cef284ca4986b189d590a6d7f, 5238714247d84a09bbc6bf5d5697baa0
I20260806 15:02:05.036161 2742 raft_consensus.cc:2726] T 1d830705329f4e17854c0113c51b2e00 P cc682dee81814201b314bd6b53858bf8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 15:02:05.677390 2743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:05.677118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.65:56061)
0806 15:02:05.677291 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:05.677337 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":76,"threads_started":1}
I20260806 15:02:05.748329 2744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:05.748084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.66:40227)
0806 15:02:05.748248 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:05.748285 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":75,"threads_started":1}
I20260806 15:02:05.885855 2745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:05.885556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.67:44605)
0806 15:02:05.885761 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:05.885805 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":99,"threads_started":1}
I20260806 15:02:06.678189 2746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:06.677936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.65:49595)
0806 15:02:06.678084 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:06.678137 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":71,"threads_started":1}
I20260806 15:02:06.749146 2748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:06.748894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.66:45161)
0806 15:02:06.749052 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:06.749097 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":68,"threads_started":1}
I20260806 15:02:06.886698 2750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:06.886401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.67:53407)
0806 15:02:06.886592 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:06.886635 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":124,"threads_started":1}
I20260806 15:02:07.679137 2751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:07.678870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.65:39705)
0806 15:02:07.679036 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:07.679082 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":70,"threads_started":1}
I20260806 15:02:07.749898 2752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:07.749659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.66:33219)
0806 15:02:07.749813 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:07.749852 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":66,"threads_started":1}
I20260806 15:02:07.887492 2753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:07.887243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.67:37457)
0806 15:02:07.887399 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:07.887442 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
I20260806 15:02:08.679894 2755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:08.679652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.65:35989)
0806 15:02:08.679810 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:08.679843 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":98,"threads_started":1}
I20260806 15:02:08.750783 2757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:08.750540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.66:60095)
0806 15:02:08.750692 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:08.750730 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
I20260806 15:02:08.888367 2758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:08.888062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.67:53541)
0806 15:02:08.888223 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:08.888270 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":67,"threads_started":1}
I20260806 15:02:09.680694 2759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:09.680466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.65:35063)
0806 15:02:09.680602 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:09.680644 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":61,"threads_started":1}
I20260806 15:02:09.751585 2761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:09.751331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.66:52315)
0806 15:02:09.751495 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:09.751538 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":83,"threads_started":1}
I20260806 15:02:09.883545 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 2097
I20260806 15:02:09.889801 2762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:09.889109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40703 (local address 127.11.8.67:44617)
0806 15:02:09.889703 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:09.889739 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.126:40703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":515,"threads_started":1}
I20260806 15:02:09.895262 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 2335
I20260806 15:02:09.904134 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 2517
2026-08-06T15:02:09Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58565 ms)
[----------] 30 tests from SecurityITest (242165 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:02:09 dist-test-slave-68xk krb5kdc[2766](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:02:09 dist-test-slave-68xk krb5kdc[2766](info): set up 2 sockets
Aug 06 15:02:09 dist-test-slave-68xk krb5kdc[2766](info): commencing operation
krb5kdc: starting...
W20260806 15:02:11.121232 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 15:02:11 dist-test-slave-68xk krb5kdc[2766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:02:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:02:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 15:02:11.250069 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:43357
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:44029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:43357
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:11.335821 2786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:11.336037 2786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:11.336061 2786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:11.337319 2786 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:11.337366 2786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:11.337383 2786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:11.337421 2786 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:11.337453 2786 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:11.339002 2786 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:44029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:43357
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:43357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:11.339270 2786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:11.339531 2786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:11.345001 2793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:11.345009 2794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:11.344993 2786 server_base.cc:1034] running on GCE node
W20260806 15:02:11.344980 2796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:11.345518 2786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:11.345786 2786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:11.346935 2786 hybrid_clock.cc:648] HybridClock initialized: now 1786028531346921 us; error 35 us; skew 500 ppm
Aug 06 15:02:11 dist-test-slave-68xk krb5kdc[2766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028531, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:11.354115 2786 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 15:02:11.354678 2786 webserver.cc:460] Webserver started at http://127.11.8.126:46191/ using document root <none> and password file <none>
I20260806 15:02:11.354915 2786 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:11.354959 2786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:11.355051 2786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:11.356179 2786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "bd5127e41dcc43a5b71e27f746d46702"
format_stamp: "Formatted at 2026-08-06 15:02:11 on dist-test-slave-68xk"
server_key: "10ba1a7c11d59433f8d15b75d4acac3c"
server_key_iv: "510baf03e9bbdba7b17f4cc4e7a2f809"
server_key_version: "encryptionkey@0"
I20260806 15:02:11.356559 2786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "bd5127e41dcc43a5b71e27f746d46702"
format_stamp: "Formatted at 2026-08-06 15:02:11 on dist-test-slave-68xk"
server_key: "10ba1a7c11d59433f8d15b75d4acac3c"
server_key_iv: "510baf03e9bbdba7b17f4cc4e7a2f809"
server_key_version: "encryptionkey@0"
I20260806 15:02:11.358011 2786 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:02:11.358855 2806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:11.359040 2786 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 15:02:11.359129 2786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "bd5127e41dcc43a5b71e27f746d46702"
format_stamp: "Formatted at 2026-08-06 15:02:11 on dist-test-slave-68xk"
server_key: "10ba1a7c11d59433f8d15b75d4acac3c"
server_key_iv: "510baf03e9bbdba7b17f4cc4e7a2f809"
server_key_version: "encryptionkey@0"
I20260806 15:02:11.359184 2786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:11.377038 2786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:11.379534 2786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:11.379653 2786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:11.383617 2786 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:43357
I20260806 15:02:11.383674 2877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:43357 every 8 connection(s)
I20260806 15:02:11.384034 2786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:02:11.385015 2879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:11.385849 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 2786
I20260806 15:02:11.385972 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:02:11.386212 11297 external_mini_cluster.cc:1428] Setting key 3a9030563bffbe19d2fb715ffe868616
I20260806 15:02:11.388568 2879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd5127e41dcc43a5b71e27f746d46702: Bootstrap starting.
I20260806 15:02:11.389183 2879 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd5127e41dcc43a5b71e27f746d46702: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:11.389503 2879 log.cc:824] T 00000000000000000000000000000000 P bd5127e41dcc43a5b71e27f746d46702: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:11.390300 2879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd5127e41dcc43a5b71e27f746d46702: No bootstrap required, opened a new log
I20260806 15:02:11.392940 2879 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd5127e41dcc43a5b71e27f746d46702 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd5127e41dcc43a5b71e27f746d46702" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 43357 } }
I20260806 15:02:11.393077 2879 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd5127e41dcc43a5b71e27f746d46702 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:11.393103 2879 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd5127e41dcc43a5b71e27f746d46702 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd5127e41dcc43a5b71e27f746d46702, State: Initialized, Role: FOLLOWER
I20260806 15:02:11.393249 2879 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd5127e41dcc43a5b71e27f746d46702 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd5127e41dcc43a5b71e27f746d46702" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 43357 } }
I20260806 15:02:11.393326 2879 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd5127e41dcc43a5b71e27f746d46702 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Aug 06 15:02:11 dist-test-slave-68xk krb5kdc[2766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:02:11.393369 2879 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd5127e41dcc43a5b71e27f746d46702 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:02:11.393414 2879 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd5127e41dcc43a5b71e27f746d46702 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:11.394115 2879 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd5127e41dcc43a5b71e27f746d46702 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd5127e41dcc43a5b71e27f746d46702" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 43357 } }
I20260806 15:02:11.394230 2879 leader_election.cc:302] T 00000000000000000000000000000000 P bd5127e41dcc43a5b71e27f746d46702 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd5127e41dcc43a5b71e27f746d46702; no voters:
I20260806 15:02:11.394464 2879 leader_election.cc:288] T 00000000000000000000000000000000 P bd5127e41dcc43a5b71e27f746d46702 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:02:11.394495 2884 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd5127e41dcc43a5b71e27f746d46702 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:11.394694 2884 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd5127e41dcc43a5b71e27f746d46702 [term 1 LEADER]: Becoming Leader. State: Replica: bd5127e41dcc43a5b71e27f746d46702, State: Running, Role: LEADER
I20260806 15:02:11.394750 2879 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd5127e41dcc43a5b71e27f746d46702 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:11.394841 2884 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd5127e41dcc43a5b71e27f746d46702 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd5127e41dcc43a5b71e27f746d46702" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 43357 } }
I20260806 15:02:11.395468 2887 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd5127e41dcc43a5b71e27f746d46702 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd5127e41dcc43a5b71e27f746d46702. Latest consensus state: current_term: 1 leader_uuid: "bd5127e41dcc43a5b71e27f746d46702" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd5127e41dcc43a5b71e27f746d46702" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 43357 } } }
I20260806 15:02:11.395565 2887 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd5127e41dcc43a5b71e27f746d46702 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:11.395603 2885 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd5127e41dcc43a5b71e27f746d46702 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd5127e41dcc43a5b71e27f746d46702" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd5127e41dcc43a5b71e27f746d46702" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 43357 } } }
I20260806 15:02:11.396021 2885 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd5127e41dcc43a5b71e27f746d46702 [sys.catalog]: This master's current role is: LEADER
W20260806 15:02:11.397176 2913 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bd5127e41dcc43a5b71e27f746d46702: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:02:11.397260 2913 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:02:11.397352 2916 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:02:11.397543 2916 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:02:11.399164 2882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:11.386827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55158 (local address 127.11.8.126:43357)
0806 15:02:11.387001 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:11.387013 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:11.387052 (+ 39us) server_negotiation.cc:407] Connection header received
0806 15:02:11.387510 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:11.387517 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:11.387564 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:11.387681 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:11.388496 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:11.389501 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:11.390537 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:11.390935 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:11.391015 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:11.393843 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:11.393848 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:11.393848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:11.393863 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:11.397438 (+ 3575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:11.397856 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:11.397859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:11.397860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:11.397970 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:11.398206 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:11.398207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:11.398207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:11.398535 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:11.398699 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:11.398816 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 15:02:11.398953 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
I20260806 15:02:11.399166 2916 catalog_manager.cc:1347] Generated new cluster ID: fcd3dfd47dfb4e8ca7a11b2dd9d498f8
I20260806 15:02:11.399251 2916 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:02:11.414394 2916 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:02:11.414969 2916 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:02:11.426530 2916 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd5127e41dcc43a5b71e27f746d46702: Generated new TSK 0
I20260806 15:02:11.426751 2916 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
I20260806 15:02:11.465288 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:0
--local_ip_for_outbound_sockets=127.11.8.65
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:43357
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:44029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:11.548844 2921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:11.549048 2921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:11.549073 2921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:11.550246 2921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:11.550313 2921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:11.550361 2921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 15:02:11.551862 2921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:44029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:43357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:11.552138 2921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:11.552383 2921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:11.557489 2930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:11.557507 2927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:11.557619 2921 server_base.cc:1034] running on GCE node
W20260806 15:02:11.557507 2928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:11.558054 2921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:11.558310 2921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:11.559458 2921 hybrid_clock.cc:648] HybridClock initialized: now 1786028531559430 us; error 66 us; skew 500 ppm
Aug 06 15:02:11 dist-test-slave-68xk krb5kdc[2766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028531, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:11.562187 2921 init.cc:377] Logged in from keytab as kudu/127.11.8.65@KRBTEST.COM (short username kudu)
I20260806 15:02:11.562696 2921 webserver.cc:460] Webserver started at http://127.11.8.65:43957/ using document root <none> and password file <none>
I20260806 15:02:11.562991 2921 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:11.563040 2921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:11.563194 2921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:11.564250 2921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-0/data/instance:
uuid: "f56babf8d8a14298b0d07df585b55406"
format_stamp: "Formatted at 2026-08-06 15:02:11 on dist-test-slave-68xk"
server_key: "7395b4ef831318b3012560bb9bb909bc"
server_key_iv: "b101bd9e59f3b0b1bd03f3e9c1695bf3"
server_key_version: "encryptionkey@0"
I20260806 15:02:11.564603 2921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance:
uuid: "f56babf8d8a14298b0d07df585b55406"
format_stamp: "Formatted at 2026-08-06 15:02:11 on dist-test-slave-68xk"
server_key: "7395b4ef831318b3012560bb9bb909bc"
server_key_iv: "b101bd9e59f3b0b1bd03f3e9c1695bf3"
server_key_version: "encryptionkey@0"
I20260806 15:02:11.565979 2921 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:02:11.566792 2941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:11.566984 2921 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 15:02:11.567078 2921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "f56babf8d8a14298b0d07df585b55406"
format_stamp: "Formatted at 2026-08-06 15:02:11 on dist-test-slave-68xk"
server_key: "7395b4ef831318b3012560bb9bb909bc"
server_key_iv: "b101bd9e59f3b0b1bd03f3e9c1695bf3"
server_key_version: "encryptionkey@0"
I20260806 15:02:11.567153 2921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:11.589520 2921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:11.591634 2921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:11.591807 2921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:11.592064 2921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:02:11.592404 2921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:02:11.592440 2921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:11.592463 2921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:02:11.592510 2921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:11.598869 2921 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:37715
I20260806 15:02:11.598964 3081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:37715 every 8 connection(s)
I20260806 15:02:11.599287 2921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
I20260806 15:02:11.601147 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 2921
I20260806 15:02:11.601271 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance
I20260806 15:02:11.601562 11297 external_mini_cluster.cc:1428] Setting key 59bf9ec5a93932992b0f4a91b1932396
Aug 06 15:02:11 dist-test-slave-68xk krb5kdc[2766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028531, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:02:11.610863 2882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:11.600151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:37165 (local address 127.11.8.126:43357)
0806 15:02:11.600213 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:11.600215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:11.600486 (+ 271us) server_negotiation.cc:407] Connection header received
0806 15:02:11.601018 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:11.601019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:11.601029 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:11.601075 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:11.602265 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:11.602866 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:11.604455 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:11.604789 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:11.604869 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:11.607591 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:11.607592 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:11.607592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:11.607600 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:11.609175 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:11.609703 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:11.609704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:11.609704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:11.609765 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:11.610070 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:11.610071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:11.610071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:11.610220 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:11.610279 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:11.610695 (+ 416us) server_negotiation.cc:299] Negotiation successful
0806 15:02:11.610790 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 15:02:11.611100 3085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:11.600120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:43357 (local address 127.11.8.65:37165)
0806 15:02:11.600347 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:11.600393 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:11.600899 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:11.601172 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:11.601176 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:11.601988 (+ 812us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:11.602186 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:11.602192 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:11.602966 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:11.602967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:11.604366 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:11.604370 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:11.604884 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:11.604884 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:11.605002 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:11.605604 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:02:11.605613 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:11.607460 (+ 1847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:11.609277 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:11.609286 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:11.609289 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:11.609620 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:11.609840 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:11.609841 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:11.609842 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:11.609994 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:11.610299 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:11.610306 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:11.610512 (+ 206us) client_negotiation.cc:769] Sending connection context
0806 15:02:11.610638 (+ 126us) client_negotiation.cc:240] Negotiation successful
0806 15:02:11.610821 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":111,"threads_started":1}
WARNING: no policy specified for kudu/127.11.8.66@KRBTEST.COM; defaulting to no policy
I20260806 15:02:11.611788 3083 heartbeater.cc:347] Connected to a master server at 127.11.8.126:43357
I20260806 15:02:11.611904 3083 heartbeater.cc:464] Registering TS with master...
I20260806 15:02:11.612115 3083 heartbeater.cc:511] Master 127.11.8.126:43357 requested a full tablet report, sending...
I20260806 15:02:11.612630 2831 ts_manager.cc:194] Registered new tserver with Master: f56babf8d8a14298b0d07df585b55406 (127.11.8.65:37715)
I20260806 15:02:11.613229 2831 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.65@KRBTEST.COM'} at 127.11.8.65:37165
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
I20260806 15:02:11.630014 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:0
--local_ip_for_outbound_sockets=127.11.8.66
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:43357
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:44029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:11.713207 3091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:11.713462 3091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:11.713502 3091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:11.714704 3091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:11.714784 3091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:11.714829 3091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 15:02:11.716346 3091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:44029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:43357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:11.716635 3091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:11.716881 3091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:11.722074 3101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:11.722131 3104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:11.722105 3098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:11.722399 3091 server_base.cc:1034] running on GCE node
I20260806 15:02:11.722661 3091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:11.722882 3091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:11.724027 3091 hybrid_clock.cc:648] HybridClock initialized: now 1786028531724010 us; error 44 us; skew 500 ppm
Aug 06 15:02:11 dist-test-slave-68xk krb5kdc[2766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028531, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:11.726687 3091 init.cc:377] Logged in from keytab as kudu/127.11.8.66@KRBTEST.COM (short username kudu)
I20260806 15:02:11.727176 3091 webserver.cc:460] Webserver started at http://127.11.8.66:45801/ using document root <none> and password file <none>
I20260806 15:02:11.727439 3091 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:11.727488 3091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:11.727665 3091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:11.728829 3091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-1/data/instance:
uuid: "65eb6f74793748729ffff5d431ea101e"
format_stamp: "Formatted at 2026-08-06 15:02:11 on dist-test-slave-68xk"
server_key: "1f1983e9a851c0e6af5524ca7d21c6c4"
server_key_iv: "657f2706122fe5241472eb907c1c69ce"
server_key_version: "encryptionkey@0"
I20260806 15:02:11.729224 3091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance:
uuid: "65eb6f74793748729ffff5d431ea101e"
format_stamp: "Formatted at 2026-08-06 15:02:11 on dist-test-slave-68xk"
server_key: "1f1983e9a851c0e6af5524ca7d21c6c4"
server_key_iv: "657f2706122fe5241472eb907c1c69ce"
server_key_version: "encryptionkey@0"
I20260806 15:02:11.730746 3091 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 15:02:11.731582 3114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:11.731796 3091 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:02:11.731894 3091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "65eb6f74793748729ffff5d431ea101e"
format_stamp: "Formatted at 2026-08-06 15:02:11 on dist-test-slave-68xk"
server_key: "1f1983e9a851c0e6af5524ca7d21c6c4"
server_key_iv: "657f2706122fe5241472eb907c1c69ce"
server_key_version: "encryptionkey@0"
I20260806 15:02:11.731976 3091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:11.747879 3091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:11.750461 3091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:11.750653 3091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:11.750923 3091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:02:11.751307 3091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:02:11.751366 3091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:11.751413 3091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:02:11.751447 3091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:11.757899 3091 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:33011
I20260806 15:02:11.757944 3263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:33011 every 8 connection(s)
I20260806 15:02:11.758273 3091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
Aug 06 15:02:11 dist-test-slave-68xk krb5kdc[2766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028531, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:02:11.765607 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 3091
I20260806 15:02:11.765744 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance
I20260806 15:02:11.765973 11297 external_mini_cluster.cc:1428] Setting key 3533a9c3827beacc857f0ee0570becee
I20260806 15:02:11.767871 3266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:11.759025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:43357 (local address 127.11.8.66:43433)
0806 15:02:11.759181 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:11.759219 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:11.759690 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:11.759892 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:11.759895 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:11.760483 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:11.760681 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:11.760685 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:11.761330 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:11.761331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:11.762160 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:11.762162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:11.762598 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:11.762598 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:11.762705 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:11.763292 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:02:11.763301 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:11.764674 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:11.766373 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:11.766381 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:11.766383 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:11.766756 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:11.766974 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:11.766974 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:11.766974 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:11.767106 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:11.767395 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:11.767397 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:11.767508 (+ 111us) client_negotiation.cc:769] Sending connection context
0806 15:02:11.767588 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 15:02:11.767711 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":95,"threads_started":1}
I20260806 15:02:11.767864 2882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:11.759044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:43433 (local address 127.11.8.126:43357)
0806 15:02:11.759092 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:11.759094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:11.759316 (+ 222us) server_negotiation.cc:407] Connection header received
0806 15:02:11.759772 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:11.759774 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:11.759784 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:11.759838 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:11.760756 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:11.761262 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:11.762249 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:11.762513 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:11.762573 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:11.764760 (+ 2187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:11.764762 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:11.764762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:11.764768 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:11.766287 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:11.766835 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:11.766837 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:11.766837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:11.766907 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:11.767188 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:11.767189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:11.767190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:11.767333 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:11.767403 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:11.767636 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 15:02:11.767725 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 15:02:11.768458 3264 heartbeater.cc:347] Connected to a master server at 127.11.8.126:43357
I20260806 15:02:11.768567 3264 heartbeater.cc:464] Registering TS with master...
I20260806 15:02:11.768774 3264 heartbeater.cc:511] Master 127.11.8.126:43357 requested a full tablet report, sending...
I20260806 15:02:11.769244 2831 ts_manager.cc:194] Registered new tserver with Master: 65eb6f74793748729ffff5d431ea101e (127.11.8.66:33011)
I20260806 15:02:11.769775 2831 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.66@KRBTEST.COM'} at 127.11.8.66:43433
WARNING: no policy specified for kudu/127.11.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
I20260806 15:02:11.793843 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:0
--local_ip_for_outbound_sockets=127.11.8.67
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:43357
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:44029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:11.878521 3269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:11.878777 3269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:11.878820 3269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:11.880028 3269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:11.880110 3269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:11.880162 3269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 15:02:11.881734 3269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:44029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:43357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:11.882014 3269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:11.882267 3269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:11.887697 3279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:11.887759 3276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:11.887737 3275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:11.887838 3269 server_base.cc:1034] running on GCE node
I20260806 15:02:11.888288 3269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:11.888546 3269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:11.889712 3269 hybrid_clock.cc:648] HybridClock initialized: now 1786028531889699 us; error 52 us; skew 500 ppm
Aug 06 15:02:11 dist-test-slave-68xk krb5kdc[2766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028531, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:11.892468 3269 init.cc:377] Logged in from keytab as kudu/127.11.8.67@KRBTEST.COM (short username kudu)
I20260806 15:02:11.892937 3269 webserver.cc:460] Webserver started at http://127.11.8.67:43423/ using document root <none> and password file <none>
I20260806 15:02:11.893195 3269 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:11.893275 3269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:11.893426 3269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:11.894562 3269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-2/data/instance:
uuid: "f2668741281c451ca8f60fc7c3174c6d"
format_stamp: "Formatted at 2026-08-06 15:02:11 on dist-test-slave-68xk"
server_key: "f2ddf4327374dceab1b40f80d22b1619"
server_key_iv: "20171b990eeef4cc22bf41cc8723752a"
server_key_version: "encryptionkey@0"
I20260806 15:02:11.894968 3269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance:
uuid: "f2668741281c451ca8f60fc7c3174c6d"
format_stamp: "Formatted at 2026-08-06 15:02:11 on dist-test-slave-68xk"
server_key: "f2ddf4327374dceab1b40f80d22b1619"
server_key_iv: "20171b990eeef4cc22bf41cc8723752a"
server_key_version: "encryptionkey@0"
I20260806 15:02:11.896634 3269 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:02:11.897476 3287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:11.897778 3269 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:02:11.897930 3269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "f2668741281c451ca8f60fc7c3174c6d"
format_stamp: "Formatted at 2026-08-06 15:02:11 on dist-test-slave-68xk"
server_key: "f2ddf4327374dceab1b40f80d22b1619"
server_key_iv: "20171b990eeef4cc22bf41cc8723752a"
server_key_version: "encryptionkey@0"
I20260806 15:02:11.898020 3269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:11.914912 3269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:11.917403 3269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:11.917577 3269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:11.917841 3269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:02:11.918177 3269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:02:11.918243 3269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:11.918288 3269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:02:11.918322 3269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:11.924638 3269 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:41177
I20260806 15:02:11.924695 3435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:41177 every 8 connection(s)
I20260806 15:02:11.925034 3269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
I20260806 15:02:11.929543 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 3269
I20260806 15:02:11.929664 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance
I20260806 15:02:11.929844 11297 external_mini_cluster.cc:1428] Setting key d8f7de18595ef6c09b9e25aaf8013c33
Aug 06 15:02:11 dist-test-slave-68xk krb5kdc[2766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028531, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:02:11.935464 2882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:11.925759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:46333 (local address 127.11.8.126:43357)
0806 15:02:11.925830 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:11.925833 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:11.926034 (+ 201us) server_negotiation.cc:407] Connection header received
0806 15:02:11.926756 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:11.926758 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:11.926771 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:11.926834 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:11.927863 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:11.928376 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:11.929453 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:11.929725 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:11.929794 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:11.932351 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:11.932353 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:11.932353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:11.932361 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:11.933733 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:11.934231 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:11.934233 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:11.934233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:11.934308 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:11.934665 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:11.934667 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:11.934667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:11.934842 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:11.934916 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:11.935256 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 15:02:11.935363 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 15:02:11.935686 3438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:11.925738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:43357 (local address 127.11.8.67:46333)
0806 15:02:11.925898 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:11.925938 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:11.926641 (+ 703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:11.926919 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:11.926928 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:11.927555 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:11.927756 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:11.927761 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:11.928481 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:11.928482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:11.929328 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:11.929330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:11.929814 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:11.929815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:11.929983 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:11.930663 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:02:11.930673 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:11.932227 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:11.933857 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:11.933866 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:11.933869 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:11.934136 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:11.934390 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:11.934390 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:11.934391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:11.934573 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:11.934908 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:11.934916 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:11.935086 (+ 170us) client_negotiation.cc:769] Sending connection context
0806 15:02:11.935206 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 15:02:11.935392 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
I20260806 15:02:11.936242 3436 heartbeater.cc:347] Connected to a master server at 127.11.8.126:43357
I20260806 15:02:11.936362 3436 heartbeater.cc:464] Registering TS with master...
I20260806 15:02:11.936581 3436 heartbeater.cc:511] Master 127.11.8.126:43357 requested a full tablet report, sending...
I20260806 15:02:11.937021 2831 ts_manager.cc:194] Registered new tserver with Master: f2668741281c451ca8f60fc7c3174c6d (127.11.8.67:41177)
I20260806 15:02:11.937422 2831 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.67@KRBTEST.COM'} at 127.11.8.67:46333
I20260806 15:02:11.941721 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:02:11.949000 2882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:11.943177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55174 (local address 127.11.8.126:43357)
0806 15:02:11.943252 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:11.943254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:11.943349 (+ 95us) server_negotiation.cc:407] Connection header received
0806 15:02:11.943374 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:11.943374 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:11.943384 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:11.943443 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:11.943966 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:11.944473 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:11.945200 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:11.945379 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:11.945442 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:11.946715 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:11.946717 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:11.946717 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:11.946724 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:11.947981 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:11.948294 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:11.948294 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:11.948295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:11.948348 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:11.948527 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:11.948528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:11.948528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:11.948675 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:11.948757 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:11.948864 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 15:02:11.948931 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 15:02:11.950636 2831 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55174:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:02:11.952230 2831 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:02:11.958881 3450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:11.954983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:37715 (local address 127.0.0.1:39002)
0806 15:02:11.955353 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:11.955362 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:11.955401 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:11.955630 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:11.955631 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:11.955633 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:11.955667 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:11.955668 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:11.956895 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:11.956896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:11.957657 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:11.957659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:11.958578 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:11.958579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:11.958658 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:11.958673 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 15:02:11.958748 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 15:02:11.958801 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":126,"threads_started":1}
I20260806 15:02:11.958920 3455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:11.955270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39002 (local address 127.11.8.65:37715)
0806 15:02:11.955471 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:11.955476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:11.955485 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:02:11.955524 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:11.955526 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:11.955541 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:11.955617 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:11.955723 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:11.956806 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:11.957754 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:11.958478 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:11.958549 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:11.958607 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:11.958784 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 15:02:11.958828 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":86,"threads_started":1}
I20260806 15:02:11.959239 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:11.954777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:33011 (local address 127.0.0.1:60706)
0806 15:02:11.954969 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:11.955011 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:11.955077 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:11.955350 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:11.955354 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:11.955357 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:11.955431 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:11.955433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:11.956841 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:11.956842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:11.958102 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:11.958104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:11.959004 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:11.959006 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:11.959081 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:11.959084 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:02:11.959118 (+ 34us) client_negotiation.cc:240] Negotiation successful
0806 15:02:11.959162 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":1,"thread_start_us":75,"threads_started":1}
I20260806 15:02:11.959362 3454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:11.954777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:41177 (local address 127.0.0.1:37868)
0806 15:02:11.955194 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:11.955205 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:11.955258 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:11.955486 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:11.955487 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:11.955489 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:11.955549 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:11.955551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:11.956629 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:11.956630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:11.957897 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:11.957900 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:11.958889 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:11.958891 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:11.958961 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:11.959145 (+ 184us) client_negotiation.cc:769] Sending connection context
0806 15:02:11.959225 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 15:02:11.959290 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":3,"thread_start_us":64,"threads_started":1}
I20260806 15:02:11.959674 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:11.954917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60706 (local address 127.11.8.66:33011)
0806 15:02:11.955127 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:11.955132 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:11.955143 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:02:11.955185 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:11.955188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:11.955204 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:11.955526 (+ 322us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:11.955608 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:11.956734 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:11.958182 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:11.958898 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:11.959418 (+ 520us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:11.959489 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:11.959515 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 15:02:11.959568 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":99,"threads_started":1}
I20260806 15:02:11.959744 3452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:11.954891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37868 (local address 127.11.8.67:41177)
0806 15:02:11.955127 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:11.955132 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:11.955263 (+ 131us) server_negotiation.cc:407] Connection header received
0806 15:02:11.955338 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:11.955341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:11.955358 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:11.955782 (+ 424us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:11.955841 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:11.956524 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:11.957983 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:11.958708 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:11.958799 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:11.958853 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:11.959616 (+ 763us) server_negotiation.cc:299] Negotiation successful
0806 15:02:11.959661 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":114,"threads_started":1}
I20260806 15:02:11.959836 3000 tablet_service.cc:1467] Processing CreateTablet for tablet 60eb3a9246a842ae8147ebd1ae4865d2 (DEFAULT_TABLE table=test-table [id=d978155ad37549e0921c5b6a3c854684]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:02:11.960235 3000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60eb3a9246a842ae8147ebd1ae4865d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:11.960356 3354 tablet_service.cc:1467] Processing CreateTablet for tablet 60eb3a9246a842ae8147ebd1ae4865d2 (DEFAULT_TABLE table=test-table [id=d978155ad37549e0921c5b6a3c854684]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:02:11.960664 3354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60eb3a9246a842ae8147ebd1ae4865d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:11.960592 3171 tablet_service.cc:1467] Processing CreateTablet for tablet 60eb3a9246a842ae8147ebd1ae4865d2 (DEFAULT_TABLE table=test-table [id=d978155ad37549e0921c5b6a3c854684]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:02:11.960907 3171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60eb3a9246a842ae8147ebd1ae4865d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:11.963862 3456 tablet_bootstrap.cc:492] T 60eb3a9246a842ae8147ebd1ae4865d2 P 65eb6f74793748729ffff5d431ea101e: Bootstrap starting.
I20260806 15:02:11.964071 3459 tablet_bootstrap.cc:492] T 60eb3a9246a842ae8147ebd1ae4865d2 P f56babf8d8a14298b0d07df585b55406: Bootstrap starting.
I20260806 15:02:11.964257 3458 tablet_bootstrap.cc:492] T 60eb3a9246a842ae8147ebd1ae4865d2 P f2668741281c451ca8f60fc7c3174c6d: Bootstrap starting.
I20260806 15:02:11.964618 3456 tablet_bootstrap.cc:654] T 60eb3a9246a842ae8147ebd1ae4865d2 P 65eb6f74793748729ffff5d431ea101e: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:11.964846 3459 tablet_bootstrap.cc:654] T 60eb3a9246a842ae8147ebd1ae4865d2 P f56babf8d8a14298b0d07df585b55406: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:11.964855 3458 tablet_bootstrap.cc:654] T 60eb3a9246a842ae8147ebd1ae4865d2 P f2668741281c451ca8f60fc7c3174c6d: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:11.965178 3456 log.cc:824] T 60eb3a9246a842ae8147ebd1ae4865d2 P 65eb6f74793748729ffff5d431ea101e: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:11.965498 3459 log.cc:824] T 60eb3a9246a842ae8147ebd1ae4865d2 P f56babf8d8a14298b0d07df585b55406: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:11.965500 3458 log.cc:824] T 60eb3a9246a842ae8147ebd1ae4865d2 P f2668741281c451ca8f60fc7c3174c6d: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:11.966048 3456 tablet_bootstrap.cc:492] T 60eb3a9246a842ae8147ebd1ae4865d2 P 65eb6f74793748729ffff5d431ea101e: No bootstrap required, opened a new log
I20260806 15:02:11.966180 3456 ts_tablet_manager.cc:1397] T 60eb3a9246a842ae8147ebd1ae4865d2 P 65eb6f74793748729ffff5d431ea101e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:02:11.966282 3458 tablet_bootstrap.cc:492] T 60eb3a9246a842ae8147ebd1ae4865d2 P f2668741281c451ca8f60fc7c3174c6d: No bootstrap required, opened a new log
I20260806 15:02:11.966365 3458 ts_tablet_manager.cc:1397] T 60eb3a9246a842ae8147ebd1ae4865d2 P f2668741281c451ca8f60fc7c3174c6d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:02:11.966703 3459 tablet_bootstrap.cc:492] T 60eb3a9246a842ae8147ebd1ae4865d2 P f56babf8d8a14298b0d07df585b55406: No bootstrap required, opened a new log
I20260806 15:02:11.966785 3459 ts_tablet_manager.cc:1397] T 60eb3a9246a842ae8147ebd1ae4865d2 P f56babf8d8a14298b0d07df585b55406: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 15:02:11.967726 3456 raft_consensus.cc:348] T 60eb3a9246a842ae8147ebd1ae4865d2 P 65eb6f74793748729ffff5d431ea101e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2668741281c451ca8f60fc7c3174c6d" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41177 } } peers { permanent_uuid: "f56babf8d8a14298b0d07df585b55406" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37715 } } peers { permanent_uuid: "65eb6f74793748729ffff5d431ea101e" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 33011 } }
I20260806 15:02:11.967892 3456 raft_consensus.cc:374] T 60eb3a9246a842ae8147ebd1ae4865d2 P 65eb6f74793748729ffff5d431ea101e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:11.967957 3456 raft_consensus.cc:729] T 60eb3a9246a842ae8147ebd1ae4865d2 P 65eb6f74793748729ffff5d431ea101e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65eb6f74793748729ffff5d431ea101e, State: Initialized, Role: FOLLOWER
I20260806 15:02:11.968094 3456 consensus_queue.cc:260] T 60eb3a9246a842ae8147ebd1ae4865d2 P 65eb6f74793748729ffff5d431ea101e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2668741281c451ca8f60fc7c3174c6d" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41177 } } peers { permanent_uuid: "f56babf8d8a14298b0d07df585b55406" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37715 } } peers { permanent_uuid: "65eb6f74793748729ffff5d431ea101e" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 33011 } }
I20260806 15:02:11.968400 3264 heartbeater.cc:503] Master 127.11.8.126:43357 was elected leader, sending a full tablet report...
I20260806 15:02:11.968405 3456 ts_tablet_manager.cc:1428] T 60eb3a9246a842ae8147ebd1ae4865d2 P 65eb6f74793748729ffff5d431ea101e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:02:11.968364 3458 raft_consensus.cc:348] T 60eb3a9246a842ae8147ebd1ae4865d2 P f2668741281c451ca8f60fc7c3174c6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2668741281c451ca8f60fc7c3174c6d" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41177 } } peers { permanent_uuid: "f56babf8d8a14298b0d07df585b55406" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37715 } } peers { permanent_uuid: "65eb6f74793748729ffff5d431ea101e" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 33011 } }
I20260806 15:02:11.968528 3458 raft_consensus.cc:374] T 60eb3a9246a842ae8147ebd1ae4865d2 P f2668741281c451ca8f60fc7c3174c6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:11.968598 3458 raft_consensus.cc:729] T 60eb3a9246a842ae8147ebd1ae4865d2 P f2668741281c451ca8f60fc7c3174c6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2668741281c451ca8f60fc7c3174c6d, State: Initialized, Role: FOLLOWER
I20260806 15:02:11.968720 3458 consensus_queue.cc:260] T 60eb3a9246a842ae8147ebd1ae4865d2 P f2668741281c451ca8f60fc7c3174c6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2668741281c451ca8f60fc7c3174c6d" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41177 } } peers { permanent_uuid: "f56babf8d8a14298b0d07df585b55406" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37715 } } peers { permanent_uuid: "65eb6f74793748729ffff5d431ea101e" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 33011 } }
I20260806 15:02:11.968780 3459 raft_consensus.cc:348] T 60eb3a9246a842ae8147ebd1ae4865d2 P f56babf8d8a14298b0d07df585b55406 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2668741281c451ca8f60fc7c3174c6d" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41177 } } peers { permanent_uuid: "f56babf8d8a14298b0d07df585b55406" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37715 } } peers { permanent_uuid: "65eb6f74793748729ffff5d431ea101e" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 33011 } }
I20260806 15:02:11.968956 3459 raft_consensus.cc:374] T 60eb3a9246a842ae8147ebd1ae4865d2 P f56babf8d8a14298b0d07df585b55406 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:11.969044 3459 raft_consensus.cc:729] T 60eb3a9246a842ae8147ebd1ae4865d2 P f56babf8d8a14298b0d07df585b55406 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f56babf8d8a14298b0d07df585b55406, State: Initialized, Role: FOLLOWER
I20260806 15:02:11.969066 3458 ts_tablet_manager.cc:1428] T 60eb3a9246a842ae8147ebd1ae4865d2 P f2668741281c451ca8f60fc7c3174c6d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 15:02:11.969182 3436 heartbeater.cc:503] Master 127.11.8.126:43357 was elected leader, sending a full tablet report...
I20260806 15:02:11.969213 3459 consensus_queue.cc:260] T 60eb3a9246a842ae8147ebd1ae4865d2 P f56babf8d8a14298b0d07df585b55406 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2668741281c451ca8f60fc7c3174c6d" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41177 } } peers { permanent_uuid: "f56babf8d8a14298b0d07df585b55406" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37715 } } peers { permanent_uuid: "65eb6f74793748729ffff5d431ea101e" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 33011 } }
I20260806 15:02:11.969525 3459 ts_tablet_manager.cc:1428] T 60eb3a9246a842ae8147ebd1ae4865d2 P f56babf8d8a14298b0d07df585b55406: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260806 15:02:11.969643 3083 heartbeater.cc:503] Master 127.11.8.126:43357 was elected leader, sending a full tablet report...
W20260806 15:02:12.008903 3265 tablet.cc:2369] T 60eb3a9246a842ae8147ebd1ae4865d2 P 65eb6f74793748729ffff5d431ea101e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 15:02:12.100067 3084 tablet.cc:2369] T 60eb3a9246a842ae8147ebd1ae4865d2 P f56babf8d8a14298b0d07df585b55406: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:02:12.113843 3465 raft_consensus.cc:482] T 60eb3a9246a842ae8147ebd1ae4865d2 P f56babf8d8a14298b0d07df585b55406 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:02:12.113966 3465 raft_consensus.cc:504] T 60eb3a9246a842ae8147ebd1ae4865d2 P f56babf8d8a14298b0d07df585b55406 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2668741281c451ca8f60fc7c3174c6d" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41177 } } peers { permanent_uuid: "f56babf8d8a14298b0d07df585b55406" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37715 } } peers { permanent_uuid: "65eb6f74793748729ffff5d431ea101e" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 33011 } }
I20260806 15:02:12.114377 3465 leader_election.cc:288] T 60eb3a9246a842ae8147ebd1ae4865d2 P f56babf8d8a14298b0d07df585b55406 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2668741281c451ca8f60fc7c3174c6d (127.11.8.67:41177), 65eb6f74793748729ffff5d431ea101e (127.11.8.66:33011)
I20260806 15:02:12.117420 3466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:12.114578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:33011 (local address 127.11.8.65:50971)
0806 15:02:12.114753 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:12.114766 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:12.114859 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:12.115032 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:12.115034 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:12.115036 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:12.115119 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:12.115121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.115770 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.115771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:12.116578 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:12.116580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.117155 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.117157 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:12.117232 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:12.117236 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:02:12.117280 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 15:02:12.117310 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":1,"thread_start_us":66,"threads_started":1}
I20260806 15:02:12.117427 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:12.114686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:50971 (local address 127.11.8.66:33011)
0806 15:02:12.114744 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:12.114746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:12.114843 (+ 97us) server_negotiation.cc:407] Connection header received
0806 15:02:12.114923 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:12.114926 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:12.114941 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:12.115012 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:12.115225 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.115675 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.116688 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.117070 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.117124 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:12.117162 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:12.117311 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 15:02:12.117357 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 15:02:12.117498 3452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:12.114714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:40109 (local address 127.11.8.67:41177)
0806 15:02:12.114760 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:12.114763 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:12.114848 (+ 85us) server_negotiation.cc:407] Connection header received
0806 15:02:12.114911 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:12.114912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:12.114926 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:12.114981 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:12.115183 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.115675 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.116694 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.117083 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.117171 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:12.117181 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:12.117382 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 15:02:12.117425 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 15:02:12.117552 3467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:12.114588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:41177 (local address 127.11.8.65:40109)
0806 15:02:12.114762 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:12.114773 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:12.114847 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:12.115010 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:12.115011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:12.115013 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:12.115108 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:12.115109 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.115767 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.115768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:12.116579 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:12.116581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.117195 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.117196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:12.117293 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:12.117300 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 15:02:12.117359 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 15:02:12.117399 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":43,"thread_start_us":108,"threads_started":1}
I20260806 15:02:12.118032 3204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60eb3a9246a842ae8147ebd1ae4865d2" candidate_uuid: "f56babf8d8a14298b0d07df585b55406" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65eb6f74793748729ffff5d431ea101e" is_pre_election: true
I20260806 15:02:12.118064 3378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60eb3a9246a842ae8147ebd1ae4865d2" candidate_uuid: "f56babf8d8a14298b0d07df585b55406" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2668741281c451ca8f60fc7c3174c6d" is_pre_election: true
I20260806 15:02:12.118216 3378 raft_consensus.cc:2454] T 60eb3a9246a842ae8147ebd1ae4865d2 P f2668741281c451ca8f60fc7c3174c6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f56babf8d8a14298b0d07df585b55406 in term 0.
I20260806 15:02:12.118216 3204 raft_consensus.cc:2454] T 60eb3a9246a842ae8147ebd1ae4865d2 P 65eb6f74793748729ffff5d431ea101e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f56babf8d8a14298b0d07df585b55406 in term 0.
I20260806 15:02:12.118464 2942 leader_election.cc:302] T 60eb3a9246a842ae8147ebd1ae4865d2 P f56babf8d8a14298b0d07df585b55406 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65eb6f74793748729ffff5d431ea101e, f56babf8d8a14298b0d07df585b55406; no voters:
I20260806 15:02:12.118620 3465 raft_consensus.cc:2781] T 60eb3a9246a842ae8147ebd1ae4865d2 P f56babf8d8a14298b0d07df585b55406 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:02:12.118717 3465 raft_consensus.cc:482] T 60eb3a9246a842ae8147ebd1ae4865d2 P f56babf8d8a14298b0d07df585b55406 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:02:12.118764 3465 raft_consensus.cc:3037] T 60eb3a9246a842ae8147ebd1ae4865d2 P f56babf8d8a14298b0d07df585b55406 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:12.119489 3465 raft_consensus.cc:504] T 60eb3a9246a842ae8147ebd1ae4865d2 P f56babf8d8a14298b0d07df585b55406 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2668741281c451ca8f60fc7c3174c6d" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41177 } } peers { permanent_uuid: "f56babf8d8a14298b0d07df585b55406" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37715 } } peers { permanent_uuid: "65eb6f74793748729ffff5d431ea101e" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 33011 } }
I20260806 15:02:12.119647 3465 leader_election.cc:288] T 60eb3a9246a842ae8147ebd1ae4865d2 P f56babf8d8a14298b0d07df585b55406 [CANDIDATE]: Term 1 election: Requested vote from peers f2668741281c451ca8f60fc7c3174c6d (127.11.8.67:41177), 65eb6f74793748729ffff5d431ea101e (127.11.8.66:33011)
I20260806 15:02:12.119804 3204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60eb3a9246a842ae8147ebd1ae4865d2" candidate_uuid: "f56babf8d8a14298b0d07df585b55406" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65eb6f74793748729ffff5d431ea101e"
I20260806 15:02:12.119844 3378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60eb3a9246a842ae8147ebd1ae4865d2" candidate_uuid: "f56babf8d8a14298b0d07df585b55406" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2668741281c451ca8f60fc7c3174c6d"
I20260806 15:02:12.119899 3204 raft_consensus.cc:3037] T 60eb3a9246a842ae8147ebd1ae4865d2 P 65eb6f74793748729ffff5d431ea101e [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:12.119956 3378 raft_consensus.cc:3037] T 60eb3a9246a842ae8147ebd1ae4865d2 P f2668741281c451ca8f60fc7c3174c6d [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:12.120656 3204 raft_consensus.cc:2454] T 60eb3a9246a842ae8147ebd1ae4865d2 P 65eb6f74793748729ffff5d431ea101e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f56babf8d8a14298b0d07df585b55406 in term 1.
I20260806 15:02:12.120738 3378 raft_consensus.cc:2454] T 60eb3a9246a842ae8147ebd1ae4865d2 P f2668741281c451ca8f60fc7c3174c6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f56babf8d8a14298b0d07df585b55406 in term 1.
I20260806 15:02:12.120880 2942 leader_election.cc:302] T 60eb3a9246a842ae8147ebd1ae4865d2 P f56babf8d8a14298b0d07df585b55406 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65eb6f74793748729ffff5d431ea101e, f56babf8d8a14298b0d07df585b55406; no voters:
I20260806 15:02:12.120973 3465 raft_consensus.cc:2781] T 60eb3a9246a842ae8147ebd1ae4865d2 P f56babf8d8a14298b0d07df585b55406 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:12.121158 3465 raft_consensus.cc:686] T 60eb3a9246a842ae8147ebd1ae4865d2 P f56babf8d8a14298b0d07df585b55406 [term 1 LEADER]: Becoming Leader. State: Replica: f56babf8d8a14298b0d07df585b55406, State: Running, Role: LEADER
I20260806 15:02:12.121295 3465 consensus_queue.cc:237] T 60eb3a9246a842ae8147ebd1ae4865d2 P f56babf8d8a14298b0d07df585b55406 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2668741281c451ca8f60fc7c3174c6d" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41177 } } peers { permanent_uuid: "f56babf8d8a14298b0d07df585b55406" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37715 } } peers { permanent_uuid: "65eb6f74793748729ffff5d431ea101e" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 33011 } }
I20260806 15:02:12.122197 2829 catalog_manager.cc:5495] T 60eb3a9246a842ae8147ebd1ae4865d2 P f56babf8d8a14298b0d07df585b55406 reported cstate change: term changed from 0 to 1, leader changed from <none> to f56babf8d8a14298b0d07df585b55406 (127.11.8.65). New cstate: current_term: 1 leader_uuid: "f56babf8d8a14298b0d07df585b55406" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2668741281c451ca8f60fc7c3174c6d" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f56babf8d8a14298b0d07df585b55406" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 37715 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "65eb6f74793748729ffff5d431ea101e" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 33011 } health_report { overall_health: UNKNOWN } } }
I20260806 15:02:12.139516 3455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:12.137330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39016 (local address 127.11.8.65:37715)
0806 15:02:12.137390 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:12.137394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:12.137408 (+ 14us) server_negotiation.cc:407] Connection header received
0806 15:02:12.137442 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:12.137443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:12.137461 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:12.137523 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:02:12.137657 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.138103 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.138899 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.139082 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.139139 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:12.139267 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:02:12.139365 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:02:12.139426 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:12.139445 (+ 19us) server_negotiation.cc:299] Negotiation successful
0806 15:02:12.139461 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 15:02:12.141357 3204 raft_consensus.cc:1261] T 60eb3a9246a842ae8147ebd1ae4865d2 P 65eb6f74793748729ffff5d431ea101e [term 1 FOLLOWER]: Refusing update from remote peer f56babf8d8a14298b0d07df585b55406: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:02:12.141428 3378 raft_consensus.cc:1261] T 60eb3a9246a842ae8147ebd1ae4865d2 P f2668741281c451ca8f60fc7c3174c6d [term 1 FOLLOWER]: Refusing update from remote peer f56babf8d8a14298b0d07df585b55406: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:02:12.141748 3470 consensus_queue.cc:1035] T 60eb3a9246a842ae8147ebd1ae4865d2 P f56babf8d8a14298b0d07df585b55406 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65eb6f74793748729ffff5d431ea101e" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 33011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:02:12.141894 3465 consensus_queue.cc:1035] T 60eb3a9246a842ae8147ebd1ae4865d2 P f56babf8d8a14298b0d07df585b55406 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2668741281c451ca8f60fc7c3174c6d" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:02:12.146832 2829 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:02:12.146970 2829 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:02:12.147819 2829 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bd5127e41dcc43a5b71e27f746d46702: Sending DeleteTablet for 3 replicas of tablet 60eb3a9246a842ae8147ebd1ae4865d2
I20260806 15:02:12.148221 3171 tablet_service.cc:1514] Processing DeleteTablet for tablet 60eb3a9246a842ae8147ebd1ae4865d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:02:12 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:60706
I20260806 15:02:12.148452 3491 tablet_replica.cc:330] T 60eb3a9246a842ae8147ebd1ae4865d2 P 65eb6f74793748729ffff5d431ea101e: stopping tablet replica
I20260806 15:02:12.148566 3491 raft_consensus.cc:2229] T 60eb3a9246a842ae8147ebd1ae4865d2 P 65eb6f74793748729ffff5d431ea101e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:02:12.148543 3354 tablet_service.cc:1514] Processing DeleteTablet for tablet 60eb3a9246a842ae8147ebd1ae4865d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:02:12 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:37868
I20260806 15:02:12.148679 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 2921
I20260806 15:02:12.148720 3491 raft_consensus.cc:2258] T 60eb3a9246a842ae8147ebd1ae4865d2 P 65eb6f74793748729ffff5d431ea101e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:02:12.148746 3000 tablet_service.cc:1514] Processing DeleteTablet for tablet 60eb3a9246a842ae8147ebd1ae4865d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:02:12 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:39002
I20260806 15:02:12.149322 3492 tablet_replica.cc:330] T 60eb3a9246a842ae8147ebd1ae4865d2 P f2668741281c451ca8f60fc7c3174c6d: stopping tablet replica
I20260806 15:02:12.149423 3492 raft_consensus.cc:2229] T 60eb3a9246a842ae8147ebd1ae4865d2 P f2668741281c451ca8f60fc7c3174c6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:02:12.149516 3492 raft_consensus.cc:2258] T 60eb3a9246a842ae8147ebd1ae4865d2 P f2668741281c451ca8f60fc7c3174c6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:02:12.154174 3492 ts_tablet_manager.cc:1905] T 60eb3a9246a842ae8147ebd1ae4865d2 P f2668741281c451ca8f60fc7c3174c6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:02:12.155824 3492 ts_tablet_manager.cc:1918] T 60eb3a9246a842ae8147ebd1ae4865d2 P f2668741281c451ca8f60fc7c3174c6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:02:12.155890 3492 log.cc:1196] T 60eb3a9246a842ae8147ebd1ae4865d2 P f2668741281c451ca8f60fc7c3174c6d: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786028287744255-11297-0/minicluster-data/ts-2/wal/wals/60eb3a9246a842ae8147ebd1ae4865d2
I20260806 15:02:12.156219 3492 ts_tablet_manager.cc:1939] T 60eb3a9246a842ae8147ebd1ae4865d2 P f2668741281c451ca8f60fc7c3174c6d: Deleting consensus metadata
I20260806 15:02:12.156672 2811 catalog_manager.cc:4841] TS f2668741281c451ca8f60fc7c3174c6d (127.11.8.67:41177): tablet 60eb3a9246a842ae8147ebd1ae4865d2 (table test-table [id=d978155ad37549e0921c5b6a3c854684]) successfully deleted
W20260806 15:02:12.158159 2811 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.65:37715 (error 108)
I20260806 15:02:12.158569 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 3091
I20260806 15:02:12.158773 3454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:12.158627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:37715 (local address 127.0.0.1:39024)
0806 15:02:12.158682 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:12.158720 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.65:37715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 15:02:12.158874 3491 ts_tablet_manager.cc:1905] T 60eb3a9246a842ae8147ebd1ae4865d2 P 65eb6f74793748729ffff5d431ea101e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 15:02:12.160414 2811 catalog_manager.cc:4584] TS f56babf8d8a14298b0d07df585b55406 (127.11.8.65:37715): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 60eb3a9246a842ae8147ebd1ae4865d2: Network error: Client connection negotiation failed: client connection to 127.11.8.65:37715: connect: Connection refused (error 111)
I20260806 15:02:12.166918 3454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:12.166774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:33011 (local address 127.0.0.1:60722)
0806 15:02:12.166829 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:12.166882 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.66:33011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 15:02:12.166929 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 3269
I20260806 15:02:12.186903 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 2786
2026-08-06T15:02:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2285 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T15:02:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:02:12Z Disabled control of system clock
I20260806 15:02:12.221940 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:37437
--webserver_interface=127.11.8.126
--webserver_port=0
--builtin_ntp_servers=127.11.8.84:43477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:37437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 15:02:12.303476 3498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:12.303701 3498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:12.303723 3498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:12.304807 3498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:12.304844 3498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:12.304857 3498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:12.304869 3498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:12.304881 3498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:12.306178 3498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:43477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:37437
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:37437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:12.306375 3498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:12.306684 3498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:12.311466 3505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:12.311501 3504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:12.311510 3508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:12.311592 3498 server_base.cc:1034] running on GCE node
I20260806 15:02:12.311995 3498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:12.312297 3498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:12.313472 3498 hybrid_clock.cc:648] HybridClock initialized: now 1786028532313449 us; error 41 us; skew 500 ppm
I20260806 15:02:12.314857 3498 webserver.cc:460] Webserver started at http://127.11.8.126:43415/ using document root <none> and password file <none>
I20260806 15:02:12.315140 3498 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:12.315231 3498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:12.315385 3498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:12.316540 3498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "1c41165dd55040c2ad3ec91092b07fce"
format_stamp: "Formatted at 2026-08-06 15:02:12 on dist-test-slave-68xk"
server_key: "b03d46886a93e03215c2bf201d88646a"
server_key_iv: "ff6c6cb70d4b3aad06afbc1446ec9631"
server_key_version: "encryptionkey@0"
I20260806 15:02:12.316908 3498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "1c41165dd55040c2ad3ec91092b07fce"
format_stamp: "Formatted at 2026-08-06 15:02:12 on dist-test-slave-68xk"
server_key: "b03d46886a93e03215c2bf201d88646a"
server_key_iv: "ff6c6cb70d4b3aad06afbc1446ec9631"
server_key_version: "encryptionkey@0"
I20260806 15:02:12.318261 3498 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260806 15:02:12.319136 3516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:12.319334 3498 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:02:12.319443 3498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "1c41165dd55040c2ad3ec91092b07fce"
format_stamp: "Formatted at 2026-08-06 15:02:12 on dist-test-slave-68xk"
server_key: "b03d46886a93e03215c2bf201d88646a"
server_key_iv: "ff6c6cb70d4b3aad06afbc1446ec9631"
server_key_version: "encryptionkey@0"
I20260806 15:02:12.319521 3498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:12.342953 3498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:12.343291 3498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:12.343451 3498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:12.347465 3498 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:37437
I20260806 15:02:12.347512 3587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:37437 every 8 connection(s)
I20260806 15:02:12.347884 3498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:02:12.348938 3588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:12.351329 3588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c41165dd55040c2ad3ec91092b07fce: Bootstrap starting.
I20260806 15:02:12.351933 3588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c41165dd55040c2ad3ec91092b07fce: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:12.352246 3588 log.cc:824] T 00000000000000000000000000000000 P 1c41165dd55040c2ad3ec91092b07fce: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:12.352953 3588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c41165dd55040c2ad3ec91092b07fce: No bootstrap required, opened a new log
I20260806 15:02:12.354609 3588 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c41165dd55040c2ad3ec91092b07fce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c41165dd55040c2ad3ec91092b07fce" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 37437 } }
I20260806 15:02:12.354749 3588 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c41165dd55040c2ad3ec91092b07fce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:12.354773 3588 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c41165dd55040c2ad3ec91092b07fce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c41165dd55040c2ad3ec91092b07fce, State: Initialized, Role: FOLLOWER
I20260806 15:02:12.354947 3588 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c41165dd55040c2ad3ec91092b07fce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c41165dd55040c2ad3ec91092b07fce" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 37437 } }
I20260806 15:02:12.355012 3588 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c41165dd55040c2ad3ec91092b07fce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:02:12.355051 3588 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c41165dd55040c2ad3ec91092b07fce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:02:12.355100 3588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c41165dd55040c2ad3ec91092b07fce [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:12.355616 3588 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c41165dd55040c2ad3ec91092b07fce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c41165dd55040c2ad3ec91092b07fce" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 37437 } }
I20260806 15:02:12.355724 3588 leader_election.cc:302] T 00000000000000000000000000000000 P 1c41165dd55040c2ad3ec91092b07fce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c41165dd55040c2ad3ec91092b07fce; no voters:
I20260806 15:02:12.355908 3588 leader_election.cc:288] T 00000000000000000000000000000000 P 1c41165dd55040c2ad3ec91092b07fce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:02:12.355979 3593 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c41165dd55040c2ad3ec91092b07fce [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:12.356175 3588 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c41165dd55040c2ad3ec91092b07fce [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:12.356216 3593 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c41165dd55040c2ad3ec91092b07fce [term 1 LEADER]: Becoming Leader. State: Replica: 1c41165dd55040c2ad3ec91092b07fce, State: Running, Role: LEADER
I20260806 15:02:12.356340 3593 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c41165dd55040c2ad3ec91092b07fce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c41165dd55040c2ad3ec91092b07fce" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 37437 } }
I20260806 15:02:12.356637 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 3498
I20260806 15:02:12.356758 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:02:12.356874 3596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c41165dd55040c2ad3ec91092b07fce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c41165dd55040c2ad3ec91092b07fce. Latest consensus state: current_term: 1 leader_uuid: "1c41165dd55040c2ad3ec91092b07fce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c41165dd55040c2ad3ec91092b07fce" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 37437 } } }
I20260806 15:02:12.356953 3596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c41165dd55040c2ad3ec91092b07fce [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:12.357009 11297 external_mini_cluster.cc:1428] Setting key 9a176ca240b9ca183fe8950a37a24e40
I20260806 15:02:12.357259 3594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c41165dd55040c2ad3ec91092b07fce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c41165dd55040c2ad3ec91092b07fce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c41165dd55040c2ad3ec91092b07fce" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 37437 } } }
I20260806 15:02:12.357321 3604 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:02:12.357326 3594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c41165dd55040c2ad3ec91092b07fce [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:12.357860 3604 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:02:12.359567 3604 catalog_manager.cc:1347] Generated new cluster ID: f69a09689384489499fd01cd7488dbaf
I20260806 15:02:12.359624 3604 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:02:12.363406 3617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:12.357701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46894 (local address 127.11.8.126:37437)
0806 15:02:12.357897 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:12.357907 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:12.357941 (+ 34us) server_negotiation.cc:407] Connection header received
0806 15:02:12.358055 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:12.358063 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:12.358131 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:12.358566 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:12.358666 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.360038 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.361154 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.361783 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.361894 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:12.362701 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:12.362705 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:12.362706 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:12.362905 (+ 199us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:12.362946 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:12.362961 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:12.363026 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:12.363123 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 15:02:12.363199 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
I20260806 15:02:12.378101 3604 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:02:12.378737 3604 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:02:12.384047 3604 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c41165dd55040c2ad3ec91092b07fce: Generated new TSK 0
I20260806 15:02:12.384275 3604 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:02:12.388651 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:0
--local_ip_for_outbound_sockets=127.11.8.65
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:37437
--builtin_ntp_servers=127.11.8.84:43477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 15:02:12.474293 3620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:12.474573 3620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:12.474603 3620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:12.475785 3620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:12.475829 3620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:12.475857 3620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 15:02:12.477222 3620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:43477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:37437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:12.477480 3620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:12.477787 3620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:12.483916 3630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:12.483932 3628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:12.483927 3626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:12.484202 3620 server_base.cc:1034] running on GCE node
I20260806 15:02:12.484413 3620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:12.484617 3620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:12.485762 3620 hybrid_clock.cc:648] HybridClock initialized: now 1786028532485750 us; error 29 us; skew 500 ppm
I20260806 15:02:12.487080 3620 webserver.cc:460] Webserver started at http://127.11.8.65:39763/ using document root <none> and password file <none>
I20260806 15:02:12.487342 3620 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:12.487391 3620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:12.487581 3620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:12.488710 3620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-0/data/instance:
uuid: "bd5dcfa543e04a63ab66edcf67703151"
format_stamp: "Formatted at 2026-08-06 15:02:12 on dist-test-slave-68xk"
server_key: "1767e42c4b2b0257ff79d15705071f54"
server_key_iv: "cbeffeb2889e54322239ebc41b7d4908"
server_key_version: "encryptionkey@0"
I20260806 15:02:12.489078 3620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance:
uuid: "bd5dcfa543e04a63ab66edcf67703151"
format_stamp: "Formatted at 2026-08-06 15:02:12 on dist-test-slave-68xk"
server_key: "1767e42c4b2b0257ff79d15705071f54"
server_key_iv: "cbeffeb2889e54322239ebc41b7d4908"
server_key_version: "encryptionkey@0"
I20260806 15:02:12.490558 3620 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:02:12.491322 3636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:12.491505 3620 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:02:12.491612 3620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "bd5dcfa543e04a63ab66edcf67703151"
format_stamp: "Formatted at 2026-08-06 15:02:12 on dist-test-slave-68xk"
server_key: "1767e42c4b2b0257ff79d15705071f54"
server_key_iv: "cbeffeb2889e54322239ebc41b7d4908"
server_key_version: "encryptionkey@0"
I20260806 15:02:12.491688 3620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:12.523665 3620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:12.524031 3620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:12.524158 3620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:12.524348 3620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:02:12.524722 3620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:02:12.524761 3620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:12.524785 3620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:02:12.524833 3620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:12.530725 3620 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:33287
I20260806 15:02:12.530771 3793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:33287 every 8 connection(s)
I20260806 15:02:12.531121 3620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
I20260806 15:02:12.534776 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 3620
I20260806 15:02:12.534889 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance
I20260806 15:02:12.535110 11297 external_mini_cluster.cc:1428] Setting key 3d4dce066101287dd553fb7d2f2d357e
I20260806 15:02:12.536188 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:0
--local_ip_for_outbound_sockets=127.11.8.66
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:37437
--builtin_ntp_servers=127.11.8.84:43477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 15:02:12.536612 3617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:12.531844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:57739 (local address 127.11.8.126:37437)
0806 15:02:12.531899 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:12.531900 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:12.532156 (+ 256us) server_negotiation.cc:407] Connection header received
0806 15:02:12.532726 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:12.532728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:12.532745 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:12.532803 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:12.533495 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.534008 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.535155 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.535396 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.535474 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:12.536332 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:12.536334 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:12.536334 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:12.536340 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:12.536362 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:12.536368 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:12.536430 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:12.536523 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 15:02:12.536551 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 15:02:12.536984 3800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:12.531816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:37437 (local address 127.11.8.65:57739)
0806 15:02:12.532015 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:12.532051 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:12.532613 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:12.532873 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:12.532877 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:12.532889 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:12.533397 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:12.533402 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.534118 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.534119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:12.535053 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:12.535056 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.535485 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.535486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:12.535631 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:12.536160 (+ 529us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:02:12.536161 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:12.536199 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:02:12.536229 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:12.536447 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:12.536455 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:12.536458 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:02:12.536528 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 15:02:12.536657 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":74,"threads_started":1}
I20260806 15:02:12.537575 3795 heartbeater.cc:347] Connected to a master server at 127.11.8.126:37437
I20260806 15:02:12.537664 3795 heartbeater.cc:464] Registering TS with master...
I20260806 15:02:12.537870 3795 heartbeater.cc:511] Master 127.11.8.126:37437 requested a full tablet report, sending...
I20260806 15:02:12.538476 3541 ts_manager.cc:194] Registered new tserver with Master: bd5dcfa543e04a63ab66edcf67703151 (127.11.8.65:33287)
I20260806 15:02:12.539166 3541 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.8.65:57739
W20260806 15:02:12.619336 3801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:12.619522 3801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:12.619544 3801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:12.620589 3801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:12.620621 3801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:12.620645 3801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 15:02:12.621907 3801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:43477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:37437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:12.622088 3801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:12.622323 3801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:12.627322 3809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:12.627336 3812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:12.627411 3810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:12.627579 3801 server_base.cc:1034] running on GCE node
I20260806 15:02:12.627830 3801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:12.628098 3801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:12.629292 3801 hybrid_clock.cc:648] HybridClock initialized: now 1786028532629274 us; error 29 us; skew 500 ppm
I20260806 15:02:12.630635 3801 webserver.cc:460] Webserver started at http://127.11.8.66:35201/ using document root <none> and password file <none>
I20260806 15:02:12.630908 3801 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:12.630985 3801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:12.631179 3801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:12.632264 3801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-1/data/instance:
uuid: "8facdfde36914d6081c5a76feef13233"
format_stamp: "Formatted at 2026-08-06 15:02:12 on dist-test-slave-68xk"
server_key: "832ee6a5eea26e28777ce80c512fee46"
server_key_iv: "28f55c5948af068ae6ba7747a9453166"
server_key_version: "encryptionkey@0"
I20260806 15:02:12.632668 3801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance:
uuid: "8facdfde36914d6081c5a76feef13233"
format_stamp: "Formatted at 2026-08-06 15:02:12 on dist-test-slave-68xk"
server_key: "832ee6a5eea26e28777ce80c512fee46"
server_key_iv: "28f55c5948af068ae6ba7747a9453166"
server_key_version: "encryptionkey@0"
I20260806 15:02:12.633994 3801 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:02:12.634899 3821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:12.635114 3801 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 15:02:12.635228 3801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "8facdfde36914d6081c5a76feef13233"
format_stamp: "Formatted at 2026-08-06 15:02:12 on dist-test-slave-68xk"
server_key: "832ee6a5eea26e28777ce80c512fee46"
server_key_iv: "28f55c5948af068ae6ba7747a9453166"
server_key_version: "encryptionkey@0"
I20260806 15:02:12.635304 3801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:12.659641 3801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:12.659941 3801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:12.660099 3801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:12.660334 3801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:02:12.660665 3801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:02:12.660723 3801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:12.660766 3801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:02:12.660794 3801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:12.666704 3801 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:36307
I20260806 15:02:12.666756 3979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:36307 every 8 connection(s)
I20260806 15:02:12.667100 3801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
I20260806 15:02:12.672462 3617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:12.667908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:44073 (local address 127.11.8.126:37437)
0806 15:02:12.667975 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:12.667978 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:12.668222 (+ 244us) server_negotiation.cc:407] Connection header received
0806 15:02:12.668755 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:12.668756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:12.668767 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:12.668819 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:12.669470 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.669958 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.671019 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.671252 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.671317 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:12.672141 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:12.672142 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:12.672143 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:12.672147 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:12.672163 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:12.672167 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:12.672212 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:12.672303 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 15:02:12.672348 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 15:02:12.672549 3983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:12.667903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:37437 (local address 127.11.8.66:44073)
0806 15:02:12.668096 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:12.668129 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:12.668655 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:12.668880 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:12.668883 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:12.668892 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:12.669395 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:12.669399 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.670060 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.670061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:12.670927 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:12.670930 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.671339 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.671340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:12.671449 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:12.671971 (+ 522us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:02:12.671971 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:12.672001 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:02:12.672023 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:12.672226 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:12.672231 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:12.672233 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 15:02:12.672287 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 15:02:12.672366 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":89,"threads_started":1}
I20260806 15:02:12.672573 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 3801
I20260806 15:02:12.672676 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance
I20260806 15:02:12.672931 11297 external_mini_cluster.cc:1428] Setting key a904cc8fc48844025d56c2267b05c46c
I20260806 15:02:12.673075 3981 heartbeater.cc:347] Connected to a master server at 127.11.8.126:37437
I20260806 15:02:12.673202 3981 heartbeater.cc:464] Registering TS with master...
I20260806 15:02:12.673415 3981 heartbeater.cc:511] Master 127.11.8.126:37437 requested a full tablet report, sending...
I20260806 15:02:12.673821 3541 ts_manager.cc:194] Registered new tserver with Master: 8facdfde36914d6081c5a76feef13233 (127.11.8.66:36307)
I20260806 15:02:12.673952 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:0
--local_ip_for_outbound_sockets=127.11.8.67
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:37437
--builtin_ntp_servers=127.11.8.84:43477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 15:02:12.674328 3541 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.8.66:44073
W20260806 15:02:12.757405 3985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:12.757630 3985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:12.757668 3985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:12.758947 3985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:12.759012 3985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:12.759055 3985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 15:02:12.760430 3985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:43477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:37437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:12.760696 3985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:12.760993 3985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:12.765939 3994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:12.765928 3997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:12.766048 3995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:12.766396 3985 server_base.cc:1034] running on GCE node
I20260806 15:02:12.766661 3985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:12.766916 3985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:12.768086 3985 hybrid_clock.cc:648] HybridClock initialized: now 1786028532768075 us; error 49 us; skew 500 ppm
I20260806 15:02:12.769346 3985 webserver.cc:460] Webserver started at http://127.11.8.67:38571/ using document root <none> and password file <none>
I20260806 15:02:12.769593 3985 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:12.769663 3985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:12.769807 3985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:12.770938 3985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-2/data/instance:
uuid: "47273b6edd8d4b34ac2b53ecc7944053"
format_stamp: "Formatted at 2026-08-06 15:02:12 on dist-test-slave-68xk"
server_key: "5f9ad6c0cdd3831086f98cfecb88eae6"
server_key_iv: "c7ed4e8632e043e96a3b9f0984b2801f"
server_key_version: "encryptionkey@0"
I20260806 15:02:12.771302 3985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance:
uuid: "47273b6edd8d4b34ac2b53ecc7944053"
format_stamp: "Formatted at 2026-08-06 15:02:12 on dist-test-slave-68xk"
server_key: "5f9ad6c0cdd3831086f98cfecb88eae6"
server_key_iv: "c7ed4e8632e043e96a3b9f0984b2801f"
server_key_version: "encryptionkey@0"
I20260806 15:02:12.772637 3985 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:02:12.773394 4005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:12.773607 3985 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:02:12.773708 3985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "47273b6edd8d4b34ac2b53ecc7944053"
format_stamp: "Formatted at 2026-08-06 15:02:12 on dist-test-slave-68xk"
server_key: "5f9ad6c0cdd3831086f98cfecb88eae6"
server_key_iv: "c7ed4e8632e043e96a3b9f0984b2801f"
server_key_version: "encryptionkey@0"
I20260806 15:02:12.773792 3985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:12.796133 3985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:12.796464 3985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:12.796625 3985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:12.796868 3985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:02:12.797217 3985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:02:12.797274 3985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:12.797310 3985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:02:12.797338 3985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:12.803550 3985 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:42081
I20260806 15:02:12.803596 4156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:42081 every 8 connection(s)
I20260806 15:02:12.803967 3985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
I20260806 15:02:12.809476 3617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:12.804794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:40923 (local address 127.11.8.126:37437)
0806 15:02:12.804873 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:12.804876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:12.805128 (+ 252us) server_negotiation.cc:407] Connection header received
0806 15:02:12.805689 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:12.805690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:12.805703 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:12.805762 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:12.806464 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.806973 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.807978 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.808183 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.808263 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:12.809148 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:12.809150 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:12.809150 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:12.809155 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:12.809180 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:12.809185 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:12.809252 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:12.809362 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 15:02:12.809395 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 15:02:12.809698 4160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:12.804744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:37437 (local address 127.11.8.67:40923)
0806 15:02:12.804976 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:12.805011 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:12.805574 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:12.805832 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:12.805843 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:12.805851 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:12.806345 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:12.806350 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.807093 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.807094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:12.807879 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:12.807881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.808271 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.808272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:12.808403 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:12.808921 (+ 518us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:02:12.808922 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:12.808951 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:02:12.808974 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:12.809255 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:12.809262 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:12.809265 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:02:12.809350 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 15:02:12.809456 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":86,"threads_started":1}
I20260806 15:02:12.809763 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 3985
I20260806 15:02:12.809863 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance
I20260806 15:02:12.810139 11297 external_mini_cluster.cc:1428] Setting key 75b0fceae7f9a93aacd3a6d4e1a2c0cc
I20260806 15:02:12.810235 4158 heartbeater.cc:347] Connected to a master server at 127.11.8.126:37437
I20260806 15:02:12.810353 4158 heartbeater.cc:464] Registering TS with master...
I20260806 15:02:12.810582 4158 heartbeater.cc:511] Master 127.11.8.126:37437 requested a full tablet report, sending...
I20260806 15:02:12.810997 3540 ts_manager.cc:194] Registered new tserver with Master: 47273b6edd8d4b34ac2b53ecc7944053 (127.11.8.67:42081)
I20260806 15:02:12.811386 3540 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.8.67:40923
I20260806 15:02:12.822006 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:02:12.826594 3617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:12.823597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46906 (local address 127.11.8.126:37437)
0806 15:02:12.823682 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:12.823684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:12.823775 (+ 91us) server_negotiation.cc:407] Connection header received
0806 15:02:12.823830 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:12.823831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:12.823845 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:12.823901 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:12.824084 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.824568 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.825352 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.825557 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.825634 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:12.826305 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:12.826306 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:12.826307 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:12.826312 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:12.826328 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:12.826333 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:12.826400 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:12.826477 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 15:02:12.826505 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 15:02:12.828325 3540 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46906:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:02:12.829994 3540 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:02:12.836967 4174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:12.832731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38472 (local address 127.11.8.66:36307)
0806 15:02:12.833130 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:12.833135 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:12.833144 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:02:12.833182 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:12.833184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:12.833199 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:12.833428 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:12.833488 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.834583 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.835710 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.836453 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.836539 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:12.836638 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:12.836838 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 15:02:12.836879 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":94,"threads_started":1}
I20260806 15:02:12.837003 4168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:12.832480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:33287 (local address 127.0.0.1:55592)
0806 15:02:12.832767 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:12.832812 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:12.832915 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:12.833204 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:12.833207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:12.833209 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:12.833294 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:12.833296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.834668 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.834669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:12.835730 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:12.835732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.836608 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.836610 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:12.836703 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:12.836721 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 15:02:12.836810 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 15:02:12.836901 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":44,"thread_start_us":62,"threads_started":1}
I20260806 15:02:12.837003 4167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:12.832481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:36307 (local address 127.0.0.1:38472)
0806 15:02:12.832860 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:12.832870 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:12.832915 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:12.833301 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:12.833302 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:12.833304 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:12.833348 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:12.833350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.834694 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.834695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:12.835614 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:12.835616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.836610 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.836611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:12.836704 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:12.836721 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 15:02:12.836810 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 15:02:12.836901 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"mutex_wait_us":7,"thread_start_us":66,"threads_started":1}
I20260806 15:02:12.837263 4173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:12.832582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55592 (local address 127.11.8.65:33287)
0806 15:02:12.832837 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:12.832842 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:12.832889 (+ 47us) server_negotiation.cc:407] Connection header received
0806 15:02:12.833002 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:12.833005 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:12.833024 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:12.833113 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:12.833380 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.834583 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.835818 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.836500 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.836565 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:12.836638 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:12.837142 (+ 504us) server_negotiation.cc:299] Negotiation successful
0806 15:02:12.837182 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":171,"threads_started":1}
I20260806 15:02:12.838195 3710 tablet_service.cc:1467] Processing CreateTablet for tablet b376093773a5438fa47e89e6c45615ee (DEFAULT_TABLE table=test-table [id=e38f60abc1754e978895ca42f0ac7279]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:02:12.838214 3888 tablet_service.cc:1467] Processing CreateTablet for tablet b376093773a5438fa47e89e6c45615ee (DEFAULT_TABLE table=test-table [id=e38f60abc1754e978895ca42f0ac7279]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:02:12.838547 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b376093773a5438fa47e89e6c45615ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:12.838554 3888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b376093773a5438fa47e89e6c45615ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:12.841264 4177 tablet_bootstrap.cc:492] T b376093773a5438fa47e89e6c45615ee P bd5dcfa543e04a63ab66edcf67703151: Bootstrap starting.
I20260806 15:02:12.841272 4178 tablet_bootstrap.cc:492] T b376093773a5438fa47e89e6c45615ee P 8facdfde36914d6081c5a76feef13233: Bootstrap starting.
I20260806 15:02:12.841701 4176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:12.832927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43484 (local address 127.11.8.67:42081)
0806 15:02:12.833270 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:12.833275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:12.833285 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:02:12.833329 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:12.833331 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:12.833348 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:12.833417 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:12.833532 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.834233 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.836688 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.837442 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.837508 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:12.837604 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:12.841534 (+ 3930us) server_negotiation.cc:299] Negotiation successful
0806 15:02:12.841600 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":85,"threads_started":1}
I20260806 15:02:12.841969 4177 tablet_bootstrap.cc:654] T b376093773a5438fa47e89e6c45615ee P bd5dcfa543e04a63ab66edcf67703151: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:12.841980 4169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:12.832493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:42081 (local address 127.0.0.1:43484)
0806 15:02:12.833020 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:12.833030 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:12.833122 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:12.833425 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:12.833426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:12.833427 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:12.833476 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:12.833477 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.834325 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.834326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:12.835726 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:12.835729 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:12.837635 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:12.837636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:12.837681 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:12.837684 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:02:12.841821 (+ 4137us) client_negotiation.cc:240] Negotiation successful
0806 15:02:12.841879 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":28,"thread_start_us":79,"threads_started":1}
I20260806 15:02:12.841981 4178 tablet_bootstrap.cc:654] T b376093773a5438fa47e89e6c45615ee P 8facdfde36914d6081c5a76feef13233: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:12.842312 4177 log.cc:824] T b376093773a5438fa47e89e6c45615ee P bd5dcfa543e04a63ab66edcf67703151: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:12.842340 4178 log.cc:824] T b376093773a5438fa47e89e6c45615ee P 8facdfde36914d6081c5a76feef13233: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:12.842801 4066 tablet_service.cc:1467] Processing CreateTablet for tablet b376093773a5438fa47e89e6c45615ee (DEFAULT_TABLE table=test-table [id=e38f60abc1754e978895ca42f0ac7279]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:02:12.843174 4177 tablet_bootstrap.cc:492] T b376093773a5438fa47e89e6c45615ee P bd5dcfa543e04a63ab66edcf67703151: No bootstrap required, opened a new log
I20260806 15:02:12.843183 4066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b376093773a5438fa47e89e6c45615ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:12.843261 4177 ts_tablet_manager.cc:1397] T b376093773a5438fa47e89e6c45615ee P bd5dcfa543e04a63ab66edcf67703151: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 15:02:12.843349 4178 tablet_bootstrap.cc:492] T b376093773a5438fa47e89e6c45615ee P 8facdfde36914d6081c5a76feef13233: No bootstrap required, opened a new log
I20260806 15:02:12.843456 4178 ts_tablet_manager.cc:1397] T b376093773a5438fa47e89e6c45615ee P 8facdfde36914d6081c5a76feef13233: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 15:02:12.845335 4177 raft_consensus.cc:348] T b376093773a5438fa47e89e6c45615ee P bd5dcfa543e04a63ab66edcf67703151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5dcfa543e04a63ab66edcf67703151" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 33287 } } peers { permanent_uuid: "8facdfde36914d6081c5a76feef13233" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36307 } } peers { permanent_uuid: "47273b6edd8d4b34ac2b53ecc7944053" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 42081 } }
I20260806 15:02:12.845496 4177 raft_consensus.cc:374] T b376093773a5438fa47e89e6c45615ee P bd5dcfa543e04a63ab66edcf67703151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:12.845543 4177 raft_consensus.cc:729] T b376093773a5438fa47e89e6c45615ee P bd5dcfa543e04a63ab66edcf67703151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd5dcfa543e04a63ab66edcf67703151, State: Initialized, Role: FOLLOWER
I20260806 15:02:12.845665 4177 consensus_queue.cc:260] T b376093773a5438fa47e89e6c45615ee P bd5dcfa543e04a63ab66edcf67703151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5dcfa543e04a63ab66edcf67703151" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 33287 } } peers { permanent_uuid: "8facdfde36914d6081c5a76feef13233" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36307 } } peers { permanent_uuid: "47273b6edd8d4b34ac2b53ecc7944053" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 42081 } }
I20260806 15:02:12.845975 4177 ts_tablet_manager.cc:1428] T b376093773a5438fa47e89e6c45615ee P bd5dcfa543e04a63ab66edcf67703151: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 15:02:12.846006 3795 heartbeater.cc:503] Master 127.11.8.126:37437 was elected leader, sending a full tablet report...
I20260806 15:02:12.846487 4178 raft_consensus.cc:348] T b376093773a5438fa47e89e6c45615ee P 8facdfde36914d6081c5a76feef13233 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5dcfa543e04a63ab66edcf67703151" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 33287 } } peers { permanent_uuid: "8facdfde36914d6081c5a76feef13233" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36307 } } peers { permanent_uuid: "47273b6edd8d4b34ac2b53ecc7944053" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 42081 } }
I20260806 15:02:12.846650 4178 raft_consensus.cc:374] T b376093773a5438fa47e89e6c45615ee P 8facdfde36914d6081c5a76feef13233 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:12.846709 4178 raft_consensus.cc:729] T b376093773a5438fa47e89e6c45615ee P 8facdfde36914d6081c5a76feef13233 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8facdfde36914d6081c5a76feef13233, State: Initialized, Role: FOLLOWER
I20260806 15:02:12.846836 4178 consensus_queue.cc:260] T b376093773a5438fa47e89e6c45615ee P 8facdfde36914d6081c5a76feef13233 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5dcfa543e04a63ab66edcf67703151" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 33287 } } peers { permanent_uuid: "8facdfde36914d6081c5a76feef13233" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36307 } } peers { permanent_uuid: "47273b6edd8d4b34ac2b53ecc7944053" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 42081 } }
I20260806 15:02:12.847150 4178 ts_tablet_manager.cc:1428] T b376093773a5438fa47e89e6c45615ee P 8facdfde36914d6081c5a76feef13233: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260806 15:02:12.847179 4183 tablet_bootstrap.cc:492] T b376093773a5438fa47e89e6c45615ee P 47273b6edd8d4b34ac2b53ecc7944053: Bootstrap starting.
I20260806 15:02:12.847164 3981 heartbeater.cc:503] Master 127.11.8.126:37437 was elected leader, sending a full tablet report...
I20260806 15:02:12.847873 4183 tablet_bootstrap.cc:654] T b376093773a5438fa47e89e6c45615ee P 47273b6edd8d4b34ac2b53ecc7944053: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:12.848206 4183 log.cc:824] T b376093773a5438fa47e89e6c45615ee P 47273b6edd8d4b34ac2b53ecc7944053: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:12.848910 4183 tablet_bootstrap.cc:492] T b376093773a5438fa47e89e6c45615ee P 47273b6edd8d4b34ac2b53ecc7944053: No bootstrap required, opened a new log
I20260806 15:02:12.848964 4183 ts_tablet_manager.cc:1397] T b376093773a5438fa47e89e6c45615ee P 47273b6edd8d4b34ac2b53ecc7944053: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:02:12.850809 4183 raft_consensus.cc:348] T b376093773a5438fa47e89e6c45615ee P 47273b6edd8d4b34ac2b53ecc7944053 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5dcfa543e04a63ab66edcf67703151" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 33287 } } peers { permanent_uuid: "8facdfde36914d6081c5a76feef13233" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36307 } } peers { permanent_uuid: "47273b6edd8d4b34ac2b53ecc7944053" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 42081 } }
I20260806 15:02:12.850929 4183 raft_consensus.cc:374] T b376093773a5438fa47e89e6c45615ee P 47273b6edd8d4b34ac2b53ecc7944053 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:12.850950 4183 raft_consensus.cc:729] T b376093773a5438fa47e89e6c45615ee P 47273b6edd8d4b34ac2b53ecc7944053 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47273b6edd8d4b34ac2b53ecc7944053, State: Initialized, Role: FOLLOWER
I20260806 15:02:12.851027 4183 consensus_queue.cc:260] T b376093773a5438fa47e89e6c45615ee P 47273b6edd8d4b34ac2b53ecc7944053 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5dcfa543e04a63ab66edcf67703151" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 33287 } } peers { permanent_uuid: "8facdfde36914d6081c5a76feef13233" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36307 } } peers { permanent_uuid: "47273b6edd8d4b34ac2b53ecc7944053" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 42081 } }
I20260806 15:02:12.851205 4183 ts_tablet_manager.cc:1428] T b376093773a5438fa47e89e6c45615ee P 47273b6edd8d4b34ac2b53ecc7944053: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:02:12.851282 4158 heartbeater.cc:503] Master 127.11.8.126:37437 was elected leader, sending a full tablet report...
W20260806 15:02:12.917716 3982 tablet.cc:2369] T b376093773a5438fa47e89e6c45615ee P 8facdfde36914d6081c5a76feef13233: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 15:02:13.031852 3796 tablet.cc:2369] T b376093773a5438fa47e89e6c45615ee P bd5dcfa543e04a63ab66edcf67703151: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 15:02:13.054630 4159 tablet.cc:2369] T b376093773a5438fa47e89e6c45615ee P 47273b6edd8d4b34ac2b53ecc7944053: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:02:13.085613 4182 raft_consensus.cc:482] T b376093773a5438fa47e89e6c45615ee P bd5dcfa543e04a63ab66edcf67703151 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:02:13.085752 4182 raft_consensus.cc:504] T b376093773a5438fa47e89e6c45615ee P bd5dcfa543e04a63ab66edcf67703151 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5dcfa543e04a63ab66edcf67703151" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 33287 } } peers { permanent_uuid: "8facdfde36914d6081c5a76feef13233" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36307 } } peers { permanent_uuid: "47273b6edd8d4b34ac2b53ecc7944053" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 42081 } }
I20260806 15:02:13.086162 4182 leader_election.cc:288] T b376093773a5438fa47e89e6c45615ee P bd5dcfa543e04a63ab66edcf67703151 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8facdfde36914d6081c5a76feef13233 (127.11.8.66:36307), 47273b6edd8d4b34ac2b53ecc7944053 (127.11.8.67:42081)
I20260806 15:02:13.089313 4189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:13.086385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:42081 (local address 127.11.8.65:46415)
0806 15:02:13.086545 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:13.086555 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:13.086648 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:13.086820 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:13.086820 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:13.086822 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:13.086904 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:13.086906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:13.087652 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:13.087653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:13.088436 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:13.088438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:13.089052 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:13.089053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:13.089123 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:13.089127 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:02:13.089182 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 15:02:13.089213 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":90,"threads_started":1}
I20260806 15:02:13.089342 4176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:13.086492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:46415 (local address 127.11.8.67:42081)
0806 15:02:13.086549 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:13.086551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:13.086663 (+ 112us) server_negotiation.cc:407] Connection header received
0806 15:02:13.086705 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:13.086708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:13.086725 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:13.086805 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:13.087001 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:13.087559 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:13.088533 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:13.088962 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:13.089022 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:13.089030 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:13.089216 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 15:02:13.089264 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 15:02:13.089443 4174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:13.086459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:59059 (local address 127.11.8.66:36307)
0806 15:02:13.086525 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:13.086528 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:13.086631 (+ 103us) server_negotiation.cc:407] Connection header received
0806 15:02:13.086695 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:13.086696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:13.086710 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:13.086768 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:13.086987 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:13.087457 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:13.088647 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:13.089013 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:13.089086 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:13.089095 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:13.089318 (+ 223us) server_negotiation.cc:299] Negotiation successful
0806 15:02:13.089366 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 15:02:13.089490 4187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:13.086359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:36307 (local address 127.11.8.65:59059)
0806 15:02:13.086535 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:13.086548 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:13.086626 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:13.086798 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:13.086799 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:13.086802 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:13.086899 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:13.086901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:13.087543 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:13.087544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:13.088548 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:13.088551 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:13.089120 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:13.089121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:13.089224 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:13.089228 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:02:13.089291 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 15:02:13.089331 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":3,"thread_start_us":91,"threads_started":1}
I20260806 15:02:13.089707 4091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b376093773a5438fa47e89e6c45615ee" candidate_uuid: "bd5dcfa543e04a63ab66edcf67703151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47273b6edd8d4b34ac2b53ecc7944053" is_pre_election: true
I20260806 15:02:13.089790 3918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b376093773a5438fa47e89e6c45615ee" candidate_uuid: "bd5dcfa543e04a63ab66edcf67703151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8facdfde36914d6081c5a76feef13233" is_pre_election: true
I20260806 15:02:13.089872 4091 raft_consensus.cc:2454] T b376093773a5438fa47e89e6c45615ee P 47273b6edd8d4b34ac2b53ecc7944053 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd5dcfa543e04a63ab66edcf67703151 in term 0.
I20260806 15:02:13.089977 3918 raft_consensus.cc:2454] T b376093773a5438fa47e89e6c45615ee P 8facdfde36914d6081c5a76feef13233 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd5dcfa543e04a63ab66edcf67703151 in term 0.
I20260806 15:02:13.090147 3643 leader_election.cc:302] T b376093773a5438fa47e89e6c45615ee P bd5dcfa543e04a63ab66edcf67703151 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47273b6edd8d4b34ac2b53ecc7944053, bd5dcfa543e04a63ab66edcf67703151; no voters:
I20260806 15:02:13.090293 4182 raft_consensus.cc:2781] T b376093773a5438fa47e89e6c45615ee P bd5dcfa543e04a63ab66edcf67703151 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:02:13.090356 4182 raft_consensus.cc:482] T b376093773a5438fa47e89e6c45615ee P bd5dcfa543e04a63ab66edcf67703151 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:02:13.090391 4182 raft_consensus.cc:3037] T b376093773a5438fa47e89e6c45615ee P bd5dcfa543e04a63ab66edcf67703151 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:13.091313 4182 raft_consensus.cc:504] T b376093773a5438fa47e89e6c45615ee P bd5dcfa543e04a63ab66edcf67703151 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5dcfa543e04a63ab66edcf67703151" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 33287 } } peers { permanent_uuid: "8facdfde36914d6081c5a76feef13233" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36307 } } peers { permanent_uuid: "47273b6edd8d4b34ac2b53ecc7944053" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 42081 } }
I20260806 15:02:13.091476 4182 leader_election.cc:288] T b376093773a5438fa47e89e6c45615ee P bd5dcfa543e04a63ab66edcf67703151 [CANDIDATE]: Term 1 election: Requested vote from peers 8facdfde36914d6081c5a76feef13233 (127.11.8.66:36307), 47273b6edd8d4b34ac2b53ecc7944053 (127.11.8.67:42081)
I20260806 15:02:13.091660 4091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b376093773a5438fa47e89e6c45615ee" candidate_uuid: "bd5dcfa543e04a63ab66edcf67703151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47273b6edd8d4b34ac2b53ecc7944053"
I20260806 15:02:13.091670 3918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b376093773a5438fa47e89e6c45615ee" candidate_uuid: "bd5dcfa543e04a63ab66edcf67703151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8facdfde36914d6081c5a76feef13233"
I20260806 15:02:13.091753 3918 raft_consensus.cc:3037] T b376093773a5438fa47e89e6c45615ee P 8facdfde36914d6081c5a76feef13233 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:13.091753 4091 raft_consensus.cc:3037] T b376093773a5438fa47e89e6c45615ee P 47273b6edd8d4b34ac2b53ecc7944053 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:13.092796 3918 raft_consensus.cc:2454] T b376093773a5438fa47e89e6c45615ee P 8facdfde36914d6081c5a76feef13233 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd5dcfa543e04a63ab66edcf67703151 in term 1.
I20260806 15:02:13.092796 4091 raft_consensus.cc:2454] T b376093773a5438fa47e89e6c45615ee P 47273b6edd8d4b34ac2b53ecc7944053 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd5dcfa543e04a63ab66edcf67703151 in term 1.
I20260806 15:02:13.093036 3639 leader_election.cc:302] T b376093773a5438fa47e89e6c45615ee P bd5dcfa543e04a63ab66edcf67703151 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8facdfde36914d6081c5a76feef13233, bd5dcfa543e04a63ab66edcf67703151; no voters:
I20260806 15:02:13.093179 4182 raft_consensus.cc:2781] T b376093773a5438fa47e89e6c45615ee P bd5dcfa543e04a63ab66edcf67703151 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:13.093379 4182 raft_consensus.cc:686] T b376093773a5438fa47e89e6c45615ee P bd5dcfa543e04a63ab66edcf67703151 [term 1 LEADER]: Becoming Leader. State: Replica: bd5dcfa543e04a63ab66edcf67703151, State: Running, Role: LEADER
I20260806 15:02:13.093482 4182 consensus_queue.cc:237] T b376093773a5438fa47e89e6c45615ee P bd5dcfa543e04a63ab66edcf67703151 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5dcfa543e04a63ab66edcf67703151" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 33287 } } peers { permanent_uuid: "8facdfde36914d6081c5a76feef13233" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36307 } } peers { permanent_uuid: "47273b6edd8d4b34ac2b53ecc7944053" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 42081 } }
I20260806 15:02:13.094334 3540 catalog_manager.cc:5495] T b376093773a5438fa47e89e6c45615ee P bd5dcfa543e04a63ab66edcf67703151 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd5dcfa543e04a63ab66edcf67703151 (127.11.8.65). New cstate: current_term: 1 leader_uuid: "bd5dcfa543e04a63ab66edcf67703151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5dcfa543e04a63ab66edcf67703151" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 33287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8facdfde36914d6081c5a76feef13233" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47273b6edd8d4b34ac2b53ecc7944053" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 42081 } health_report { overall_health: UNKNOWN } } }
I20260806 15:02:13.119573 4173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:13.117226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55596 (local address 127.11.8.65:33287)
0806 15:02:13.117309 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:13.117311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:13.117319 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:02:13.117355 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:13.117356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:13.117376 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:13.117439 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:02:13.117604 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:13.118059 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:13.118880 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:13.119065 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:13.119117 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:13.119261 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:02:13.119359 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:02:13.119420 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:13.119449 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 15:02:13.119467 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 15:02:13.121577 3918 raft_consensus.cc:1261] T b376093773a5438fa47e89e6c45615ee P 8facdfde36914d6081c5a76feef13233 [term 1 FOLLOWER]: Refusing update from remote peer bd5dcfa543e04a63ab66edcf67703151: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:02:13.121634 4091 raft_consensus.cc:1261] T b376093773a5438fa47e89e6c45615ee P 47273b6edd8d4b34ac2b53ecc7944053 [term 1 FOLLOWER]: Refusing update from remote peer bd5dcfa543e04a63ab66edcf67703151: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:02:13.122130 4182 consensus_queue.cc:1035] T b376093773a5438fa47e89e6c45615ee P bd5dcfa543e04a63ab66edcf67703151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8facdfde36914d6081c5a76feef13233" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 36307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:02:13.122287 4190 consensus_queue.cc:1035] T b376093773a5438fa47e89e6c45615ee P bd5dcfa543e04a63ab66edcf67703151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47273b6edd8d4b34ac2b53ecc7944053" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 42081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:02:13.127558 3541 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:02:13.127658 3541 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:02:13.128513 3541 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c41165dd55040c2ad3ec91092b07fce: Sending DeleteTablet for 3 replicas of tablet b376093773a5438fa47e89e6c45615ee
I20260806 15:02:13.128919 4066 tablet_service.cc:1514] Processing DeleteTablet for tablet b376093773a5438fa47e89e6c45615ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:02:13 UTC) from {username='slave'} at 127.0.0.1:43484
I20260806 15:02:13.128919 3710 tablet_service.cc:1514] Processing DeleteTablet for tablet b376093773a5438fa47e89e6c45615ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:02:13 UTC) from {username='slave'} at 127.0.0.1:55592
I20260806 15:02:13.129237 4204 tablet_replica.cc:330] T b376093773a5438fa47e89e6c45615ee P 47273b6edd8d4b34ac2b53ecc7944053: stopping tablet replica
I20260806 15:02:13.129226 3888 tablet_service.cc:1514] Processing DeleteTablet for tablet b376093773a5438fa47e89e6c45615ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:02:13 UTC) from {username='slave'} at 127.0.0.1:38472
I20260806 15:02:13.129356 4204 raft_consensus.cc:2229] T b376093773a5438fa47e89e6c45615ee P 47273b6edd8d4b34ac2b53ecc7944053 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:02:13.129490 4204 raft_consensus.cc:2258] T b376093773a5438fa47e89e6c45615ee P 47273b6edd8d4b34ac2b53ecc7944053 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:02:13.129693 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 3620
I20260806 15:02:13.129966 4204 ts_tablet_manager.cc:1905] T b376093773a5438fa47e89e6c45615ee P 47273b6edd8d4b34ac2b53ecc7944053: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:02:13.132107 4204 ts_tablet_manager.cc:1918] T b376093773a5438fa47e89e6c45615ee P 47273b6edd8d4b34ac2b53ecc7944053: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:02:13.132174 4204 log.cc:1196] T b376093773a5438fa47e89e6c45615ee P 47273b6edd8d4b34ac2b53ecc7944053: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-2/wal/wals/b376093773a5438fa47e89e6c45615ee
I20260806 15:02:13.132378 4204 ts_tablet_manager.cc:1939] T b376093773a5438fa47e89e6c45615ee P 47273b6edd8d4b34ac2b53ecc7944053: Deleting consensus metadata
I20260806 15:02:13.140120 4206 tablet_replica.cc:330] T b376093773a5438fa47e89e6c45615ee P 8facdfde36914d6081c5a76feef13233: stopping tablet replica
I20260806 15:02:13.140239 4206 raft_consensus.cc:2229] T b376093773a5438fa47e89e6c45615ee P 8facdfde36914d6081c5a76feef13233 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:02:13.140333 4206 raft_consensus.cc:2258] T b376093773a5438fa47e89e6c45615ee P 8facdfde36914d6081c5a76feef13233 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:02:13.143424 3520 catalog_manager.cc:4841] TS 47273b6edd8d4b34ac2b53ecc7944053 (127.11.8.67:42081): tablet b376093773a5438fa47e89e6c45615ee (table test-table [id=e38f60abc1754e978895ca42f0ac7279]) successfully deleted
I20260806 15:02:13.144399 4206 ts_tablet_manager.cc:1905] T b376093773a5438fa47e89e6c45615ee P 8facdfde36914d6081c5a76feef13233: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:02:13.146023 4206 ts_tablet_manager.cc:1918] T b376093773a5438fa47e89e6c45615ee P 8facdfde36914d6081c5a76feef13233: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:02:13.146111 4206 log.cc:1196] T b376093773a5438fa47e89e6c45615ee P 8facdfde36914d6081c5a76feef13233: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786028287744255-11297-0/minicluster-data/ts-1/wal/wals/b376093773a5438fa47e89e6c45615ee
I20260806 15:02:13.146406 4206 ts_tablet_manager.cc:1939] T b376093773a5438fa47e89e6c45615ee P 8facdfde36914d6081c5a76feef13233: Deleting consensus metadata
I20260806 15:02:13.146975 3519 catalog_manager.cc:4841] TS 8facdfde36914d6081c5a76feef13233 (127.11.8.66:36307): tablet b376093773a5438fa47e89e6c45615ee (table test-table [id=e38f60abc1754e978895ca42f0ac7279]) successfully deleted
W20260806 15:02:13.148718 3521 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.65:33287 (error 108)
I20260806 15:02:13.148933 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 3801
I20260806 15:02:13.151587 4169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:13.149079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:33287 (local address 127.0.0.1:55602)
0806 15:02:13.151509 (+ 2430us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:13.151558 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.65:33287: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2407}
W20260806 15:02:13.151721 3521 catalog_manager.cc:4584] TS bd5dcfa543e04a63ab66edcf67703151 (127.11.8.65:33287): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b376093773a5438fa47e89e6c45615ee: Network error: Client connection negotiation failed: client connection to 127.11.8.65:33287: connect: Connection refused (error 111)
I20260806 15:02:13.161109 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 3985
I20260806 15:02:13.171168 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 3498
2026-08-06T15:02:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (980 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:02:13 dist-test-slave-68xk krb5kdc[4211](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:02:13 dist-test-slave-68xk krb5kdc[4211](info): set up 2 sockets
Aug 06 15:02:13 dist-test-slave-68xk krb5kdc[4211](info): commencing operation
krb5kdc: starting...
W20260806 15:02:14.392640 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 15:02:14 dist-test-slave-68xk krb5kdc[4211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:02:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:02:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 15:02:14.516899 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:40051
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:37457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:40051
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:14.600564 4226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:14.600775 4226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:14.600796 4226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:14.601883 4226 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:14.601922 4226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:14.601938 4226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:14.601950 4226 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:14.601962 4226 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:14.603407 4226 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:37457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:40051
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:40051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:14.603609 4226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:14.603778 4226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:14.608757 4232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:14.608816 4236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:14.608960 4226 server_base.cc:1034] running on GCE node
W20260806 15:02:14.608781 4234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:14.609558 4226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:14.609882 4226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:14.611069 4226 hybrid_clock.cc:648] HybridClock initialized: now 1786028534611044 us; error 44 us; skew 500 ppm
Aug 06 15:02:14 dist-test-slave-68xk krb5kdc[4211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028534, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:14.613766 4226 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 15:02:14.614190 4226 webserver.cc:460] Webserver started at http://127.11.8.126:42757/ using document root <none> and password file <none>
I20260806 15:02:14.614468 4226 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:14.614532 4226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:14.614642 4226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:14.615734 4226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "5b53028b1cb34f68a50ab2cf8c507c87"
format_stamp: "Formatted at 2026-08-06 15:02:14 on dist-test-slave-68xk"
server_key: "272b179a5195c246a9c9ab83c5e06165"
server_key_iv: "5a24e45d7f1497e8995431c9bebf1631"
server_key_version: "encryptionkey@0"
I20260806 15:02:14.616065 4226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "5b53028b1cb34f68a50ab2cf8c507c87"
format_stamp: "Formatted at 2026-08-06 15:02:14 on dist-test-slave-68xk"
server_key: "272b179a5195c246a9c9ab83c5e06165"
server_key_iv: "5a24e45d7f1497e8995431c9bebf1631"
server_key_version: "encryptionkey@0"
I20260806 15:02:14.617477 4226 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:02:14.618279 4251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:14.618525 4226 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:02:14.618645 4226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "5b53028b1cb34f68a50ab2cf8c507c87"
format_stamp: "Formatted at 2026-08-06 15:02:14 on dist-test-slave-68xk"
server_key: "272b179a5195c246a9c9ab83c5e06165"
server_key_iv: "5a24e45d7f1497e8995431c9bebf1631"
server_key_version: "encryptionkey@0"
I20260806 15:02:14.618742 4226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:14.638746 4226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:14.640831 4226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:14.641022 4226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:14.645640 4226 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:40051
I20260806 15:02:14.645705 4323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:40051 every 8 connection(s)
I20260806 15:02:14.646111 4226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:02:14.647054 4324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:14.649333 4324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b53028b1cb34f68a50ab2cf8c507c87: Bootstrap starting.
I20260806 15:02:14.649948 4324 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b53028b1cb34f68a50ab2cf8c507c87: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:14.650242 4324 log.cc:824] T 00000000000000000000000000000000 P 5b53028b1cb34f68a50ab2cf8c507c87: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:14.650985 4324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b53028b1cb34f68a50ab2cf8c507c87: No bootstrap required, opened a new log
I20260806 15:02:14.651746 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 4226
I20260806 15:02:14.651898 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:02:14.652144 11297 external_mini_cluster.cc:1428] Setting key 0d013db07bbfe86c83e381a9efca4b4f
I20260806 15:02:14.653031 4324 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b53028b1cb34f68a50ab2cf8c507c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b53028b1cb34f68a50ab2cf8c507c87" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40051 } }
I20260806 15:02:14.653187 4324 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b53028b1cb34f68a50ab2cf8c507c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:14.653252 4324 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b53028b1cb34f68a50ab2cf8c507c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b53028b1cb34f68a50ab2cf8c507c87, State: Initialized, Role: FOLLOWER
I20260806 15:02:14.653442 4324 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b53028b1cb34f68a50ab2cf8c507c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b53028b1cb34f68a50ab2cf8c507c87" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40051 } }
I20260806 15:02:14.653545 4324 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b53028b1cb34f68a50ab2cf8c507c87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:02:14.653605 4324 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b53028b1cb34f68a50ab2cf8c507c87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:02:14.653659 4324 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b53028b1cb34f68a50ab2cf8c507c87 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:14.654191 4324 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b53028b1cb34f68a50ab2cf8c507c87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b53028b1cb34f68a50ab2cf8c507c87" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40051 } }
I20260806 15:02:14.654312 4324 leader_election.cc:302] T 00000000000000000000000000000000 P 5b53028b1cb34f68a50ab2cf8c507c87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b53028b1cb34f68a50ab2cf8c507c87; no voters:
I20260806 15:02:14.654511 4324 leader_election.cc:288] T 00000000000000000000000000000000 P 5b53028b1cb34f68a50ab2cf8c507c87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:02:14.654546 4334 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b53028b1cb34f68a50ab2cf8c507c87 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:14.654663 4334 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b53028b1cb34f68a50ab2cf8c507c87 [term 1 LEADER]: Becoming Leader. State: Replica: 5b53028b1cb34f68a50ab2cf8c507c87, State: Running, Role: LEADER
I20260806 15:02:14.654784 4324 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b53028b1cb34f68a50ab2cf8c507c87 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:14.654803 4334 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b53028b1cb34f68a50ab2cf8c507c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b53028b1cb34f68a50ab2cf8c507c87" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40051 } }
I20260806 15:02:14.655161 4334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b53028b1cb34f68a50ab2cf8c507c87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b53028b1cb34f68a50ab2cf8c507c87. Latest consensus state: current_term: 1 leader_uuid: "5b53028b1cb34f68a50ab2cf8c507c87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b53028b1cb34f68a50ab2cf8c507c87" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40051 } } }
I20260806 15:02:14.655175 4338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b53028b1cb34f68a50ab2cf8c507c87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b53028b1cb34f68a50ab2cf8c507c87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b53028b1cb34f68a50ab2cf8c507c87" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40051 } } }
I20260806 15:02:14.655318 4334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b53028b1cb34f68a50ab2cf8c507c87 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:14.655529 4338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b53028b1cb34f68a50ab2cf8c507c87 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:14.655720 4345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:02:14.656294 4345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:02:14.657924 4345 catalog_manager.cc:1347] Generated new cluster ID: 42324708503d46618b933e290e0290ec
I20260806 15:02:14.657982 4345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 15:02:14 dist-test-slave-68xk krb5kdc[4211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:02:14.663182 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:14.652705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40354 (local address 127.11.8.126:40051)
0806 15:02:14.652911 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:14.652919 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:14.652946 (+ 27us) server_negotiation.cc:407] Connection header received
0806 15:02:14.653050 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:14.653062 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:14.653107 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:14.653254 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:14.654157 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:14.655697 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:14.656921 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:14.657557 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:14.657675 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:14.660079 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:14.660083 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:14.660083 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:14.660101 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:14.661635 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:14.662011 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:14.662014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:14.662014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:14.662111 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:14.662336 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:14.662337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:14.662337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:14.662619 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:14.662811 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:14.662876 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 15:02:14.663012 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":110,"threads_started":1}
I20260806 15:02:14.676143 4345 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:02:14.676950 4345 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:02:14.688392 4345 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b53028b1cb34f68a50ab2cf8c507c87: Generated new TSK 0
I20260806 15:02:14.688663 4345 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
I20260806 15:02:14.727438 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:0
--local_ip_for_outbound_sockets=127.11.8.65
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:40051
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:37457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:14.836639 4361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:14.836854 4361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:14.836879 4361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:14.837956 4361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:14.837989 4361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:14.838016 4361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 15:02:14.839398 4361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:37457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:40051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:14.839576 4361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:14.839747 4361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:14.844838 4372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:14.845052 4361 server_base.cc:1034] running on GCE node
W20260806 15:02:14.844852 4368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:14.844852 4369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:14.845392 4361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:14.845613 4361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:14.846935 4361 hybrid_clock.cc:648] HybridClock initialized: now 1786028534846923 us; error 33 us; skew 500 ppm
Aug 06 15:02:14 dist-test-slave-68xk krb5kdc[4211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028534, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:14.849843 4361 init.cc:377] Logged in from keytab as kudu/127.11.8.65@KRBTEST.COM (short username kudu)
I20260806 15:02:14.850353 4361 webserver.cc:460] Webserver started at http://127.11.8.65:33289/ using document root <none> and password file <none>
I20260806 15:02:14.850628 4361 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:14.850682 4361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:14.850781 4361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:14.851873 4361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-0/data/instance:
uuid: "aa377ec71efd49b19cfd7ecd17e3686b"
format_stamp: "Formatted at 2026-08-06 15:02:14 on dist-test-slave-68xk"
server_key: "bfb522e26de1e82614645617eb25c8d2"
server_key_iv: "2e19fe25cf2d10808457b5ae2d85460e"
server_key_version: "encryptionkey@0"
I20260806 15:02:14.852226 4361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance:
uuid: "aa377ec71efd49b19cfd7ecd17e3686b"
format_stamp: "Formatted at 2026-08-06 15:02:14 on dist-test-slave-68xk"
server_key: "bfb522e26de1e82614645617eb25c8d2"
server_key_iv: "2e19fe25cf2d10808457b5ae2d85460e"
server_key_version: "encryptionkey@0"
I20260806 15:02:14.853623 4361 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:02:14.854490 4381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:14.854699 4361 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:02:14.854807 4361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "aa377ec71efd49b19cfd7ecd17e3686b"
format_stamp: "Formatted at 2026-08-06 15:02:14 on dist-test-slave-68xk"
server_key: "bfb522e26de1e82614645617eb25c8d2"
server_key_iv: "2e19fe25cf2d10808457b5ae2d85460e"
server_key_version: "encryptionkey@0"
I20260806 15:02:14.854892 4361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:14.873136 4361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:14.875756 4361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:14.875936 4361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:14.876235 4361 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:02:14.876600 4361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:02:14.876638 4361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:14.876663 4361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:02:14.876677 4361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:14.883121 4361 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:34627
I20260806 15:02:14.883190 4545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:34627 every 8 connection(s)
I20260806 15:02:14.883572 4361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
Aug 06 15:02:14 dist-test-slave-68xk krb5kdc[4211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028534, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:02:14.893522 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 4361
I20260806 15:02:14.893687 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance
I20260806 15:02:14.893839 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:14.884416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:45841 (local address 127.11.8.126:40051)
0806 15:02:14.884464 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:14.884466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:14.884701 (+ 235us) server_negotiation.cc:407] Connection header received
0806 15:02:14.885240 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:14.885241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:14.885255 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:14.885321 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:14.886346 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:14.886934 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:14.887979 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:14.888182 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:14.888251 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:14.890700 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:14.890702 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:14.890702 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:14.890709 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:14.892386 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:14.892774 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:14.892775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:14.892775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:14.892826 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:14.893090 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:14.893091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:14.893092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:14.893220 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:14.893289 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:14.893625 (+ 336us) server_negotiation.cc:299] Negotiation successful
0806 15:02:14.893736 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 15:02:14.893955 11297 external_mini_cluster.cc:1428] Setting key 959f08c847cbc20c3e4e7c3dc10fe2f8
I20260806 15:02:14.893998 4548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:14.884373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40051 (local address 127.11.8.65:45841)
0806 15:02:14.884550 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:14.884597 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:14.885147 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:14.885380 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:14.885384 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:14.886015 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:14.886232 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:14.886237 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:14.887045 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:14.887046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:14.887889 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:14.887892 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:14.888296 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:14.888298 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:14.888417 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:14.889053 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:02:14.889063 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:14.890606 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:14.892464 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:14.892472 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:14.892474 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:14.892705 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:14.892887 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:14.892888 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:14.892888 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:14.892992 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:14.893320 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:14.893328 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:14.893455 (+ 127us) client_negotiation.cc:769] Sending connection context
0806 15:02:14.893556 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 15:02:14.893764 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":102,"threads_started":1}
I20260806 15:02:14.894750 4546 heartbeater.cc:347] Connected to a master server at 127.11.8.126:40051
I20260806 15:02:14.894886 4546 heartbeater.cc:464] Registering TS with master...
I20260806 15:02:14.895107 4546 heartbeater.cc:511] Master 127.11.8.126:40051 requested a full tablet report, sending...
I20260806 15:02:14.895867 4274 ts_manager.cc:194] Registered new tserver with Master: aa377ec71efd49b19cfd7ecd17e3686b (127.11.8.65:34627)
I20260806 15:02:14.896618 4274 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.65@KRBTEST.COM'} at 127.11.8.65:45841
WARNING: no policy specified for kudu/127.11.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
I20260806 15:02:14.921823 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:0
--local_ip_for_outbound_sockets=127.11.8.66
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:40051
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:37457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:15.017493 4553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:15.017750 4553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:15.017796 4553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:15.019181 4553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:15.019254 4553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:15.019306 4553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 15:02:15.020955 4553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:37457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:40051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:15.021261 4553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:15.021575 4553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:15.027000 4558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:15.027031 4560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:15.027081 4553 server_base.cc:1034] running on GCE node
W20260806 15:02:15.027096 4563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:15.027575 4553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:15.027863 4553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:15.029035 4553 hybrid_clock.cc:648] HybridClock initialized: now 1786028535029009 us; error 41 us; skew 500 ppm
Aug 06 15:02:15 dist-test-slave-68xk krb5kdc[4211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028535, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:15.037447 4553 init.cc:377] Logged in from keytab as kudu/127.11.8.66@KRBTEST.COM (short username kudu)
I20260806 15:02:15.038012 4553 webserver.cc:460] Webserver started at http://127.11.8.66:35145/ using document root <none> and password file <none>
I20260806 15:02:15.038307 4553 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:15.038389 4553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:15.038590 4553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:15.039790 4553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-1/data/instance:
uuid: "3d05bb57b47d45e3a4206536abc58f89"
format_stamp: "Formatted at 2026-08-06 15:02:15 on dist-test-slave-68xk"
server_key: "7a363b4a999028681379f9b2afe69e39"
server_key_iv: "96cdf00850bb9db419f80c14c2fed5d9"
server_key_version: "encryptionkey@0"
I20260806 15:02:15.040196 4553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance:
uuid: "3d05bb57b47d45e3a4206536abc58f89"
format_stamp: "Formatted at 2026-08-06 15:02:15 on dist-test-slave-68xk"
server_key: "7a363b4a999028681379f9b2afe69e39"
server_key_iv: "96cdf00850bb9db419f80c14c2fed5d9"
server_key_version: "encryptionkey@0"
I20260806 15:02:15.041695 4553 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:02:15.042625 4573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:15.042840 4553 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:02:15.042961 4553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "3d05bb57b47d45e3a4206536abc58f89"
format_stamp: "Formatted at 2026-08-06 15:02:15 on dist-test-slave-68xk"
server_key: "7a363b4a999028681379f9b2afe69e39"
server_key_iv: "96cdf00850bb9db419f80c14c2fed5d9"
server_key_version: "encryptionkey@0"
I20260806 15:02:15.043048 4553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:15.069859 4553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:15.072366 4553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:15.072636 4553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:15.072952 4553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:02:15.073313 4553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:02:15.073374 4553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:15.073418 4553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:02:15.073448 4553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:15.080263 4553 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:40027
I20260806 15:02:15.080315 4729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:40027 every 8 connection(s)
I20260806 15:02:15.080641 4553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
Aug 06 15:02:15 dist-test-slave-68xk krb5kdc[4211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028535, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:02:15.087873 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 4553
I20260806 15:02:15.088001 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance
I20260806 15:02:15.088280 11297 external_mini_cluster.cc:1428] Setting key 501c1160b3ba02423953d39885ccb413
I20260806 15:02:15.090389 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:15.081398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:57579 (local address 127.11.8.126:40051)
0806 15:02:15.081447 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:15.081449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:15.081673 (+ 224us) server_negotiation.cc:407] Connection header received
0806 15:02:15.082204 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:15.082205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:15.082218 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:15.082269 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:15.083305 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.083818 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.084844 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.085061 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.085144 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:15.087381 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:15.087383 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:15.087383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:15.087390 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:15.088922 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:15.089394 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:15.089395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:15.089396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:15.089458 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:15.089747 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:15.089748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:15.089748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:15.089860 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:15.089917 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:15.090241 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 15:02:15.090322 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 15:02:15.090713 4732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:15.081356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40051 (local address 127.11.8.66:57579)
0806 15:02:15.081546 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:15.081583 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:15.082094 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:15.082342 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:15.082347 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:15.082990 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:15.083183 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:15.083187 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.083913 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.083914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:15.084739 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:15.084742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.085150 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.085151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:15.085299 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:15.085785 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:02:15.085794 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:15.087281 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:15.088982 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:15.088990 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:15.088992 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:15.089313 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:15.089531 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:15.089532 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:15.089532 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:15.089674 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:15.089929 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:15.089935 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:15.090102 (+ 167us) client_negotiation.cc:769] Sending connection context
0806 15:02:15.090214 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 15:02:15.090401 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":94,"threads_started":1}
I20260806 15:02:15.091266 4730 heartbeater.cc:347] Connected to a master server at 127.11.8.126:40051
I20260806 15:02:15.091384 4730 heartbeater.cc:464] Registering TS with master...
I20260806 15:02:15.091565 4730 heartbeater.cc:511] Master 127.11.8.126:40051 requested a full tablet report, sending...
I20260806 15:02:15.091956 4274 ts_manager.cc:194] Registered new tserver with Master: 3d05bb57b47d45e3a4206536abc58f89 (127.11.8.66:40027)
I20260806 15:02:15.092357 4274 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.66@KRBTEST.COM'} at 127.11.8.66:57579
WARNING: no policy specified for kudu/127.11.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
I20260806 15:02:15.121336 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:0
--local_ip_for_outbound_sockets=127.11.8.67
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:40051
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:37457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:15.215520 4738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:15.215725 4738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:15.215749 4738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:15.216841 4738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:15.216884 4738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:15.216910 4738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 15:02:15.218396 4738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:37457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:40051
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:15.218611 4738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:15.218784 4738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:15.223975 4746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:15.223982 4749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:15.224025 4744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:15.224225 4738 server_base.cc:1034] running on GCE node
I20260806 15:02:15.224423 4738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:15.224624 4738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:15.225808 4738 hybrid_clock.cc:648] HybridClock initialized: now 1786028535225794 us; error 31 us; skew 500 ppm
Aug 06 15:02:15 dist-test-slave-68xk krb5kdc[4211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028535, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:15.228559 4738 init.cc:377] Logged in from keytab as kudu/127.11.8.67@KRBTEST.COM (short username kudu)
I20260806 15:02:15.229039 4738 webserver.cc:460] Webserver started at http://127.11.8.67:35427/ using document root <none> and password file <none>
I20260806 15:02:15.229324 4738 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:15.229405 4738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:15.229571 4738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:15.230782 4738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-2/data/instance:
uuid: "de3141875c6849df93e652c86b5196a5"
format_stamp: "Formatted at 2026-08-06 15:02:15 on dist-test-slave-68xk"
server_key: "066e60498d9a3f8c2367524552701b6a"
server_key_iv: "e4621d33837ec165f3aa7e1d369e010f"
server_key_version: "encryptionkey@0"
I20260806 15:02:15.231175 4738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance:
uuid: "de3141875c6849df93e652c86b5196a5"
format_stamp: "Formatted at 2026-08-06 15:02:15 on dist-test-slave-68xk"
server_key: "066e60498d9a3f8c2367524552701b6a"
server_key_iv: "e4621d33837ec165f3aa7e1d369e010f"
server_key_version: "encryptionkey@0"
I20260806 15:02:15.232674 4738 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:02:15.233536 4759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:15.233753 4738 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:02:15.233872 4738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "de3141875c6849df93e652c86b5196a5"
format_stamp: "Formatted at 2026-08-06 15:02:15 on dist-test-slave-68xk"
server_key: "066e60498d9a3f8c2367524552701b6a"
server_key_iv: "e4621d33837ec165f3aa7e1d369e010f"
server_key_version: "encryptionkey@0"
I20260806 15:02:15.233953 4738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:15.248153 4738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:15.250608 4738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:15.250794 4738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:15.250981 4738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:02:15.251308 4738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:02:15.251343 4738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:15.251396 4738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:02:15.251430 4738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:15.257771 4738 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:39459
I20260806 15:02:15.257840 4905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:39459 every 8 connection(s)
I20260806 15:02:15.258167 4738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
Aug 06 15:02:15 dist-test-slave-68xk krb5kdc[4211](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028535, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:02:15.267712 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:15.258966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:42007 (local address 127.11.8.126:40051)
0806 15:02:15.259036 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:15.259038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:15.259276 (+ 238us) server_negotiation.cc:407] Connection header received
0806 15:02:15.259828 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:15.259829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:15.259844 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:15.259909 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:15.260916 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.261439 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.262544 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.262754 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.262833 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:15.265037 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:15.265039 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:15.265039 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:15.265047 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:15.266402 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:15.266859 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:15.266861 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:15.266861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:15.266919 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:15.267138 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:15.267139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:15.267139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:15.267274 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:15.267329 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:15.267573 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 15:02:15.267642 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 15:02:15.267918 4908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:15.258983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40051 (local address 127.11.8.67:42007)
0806 15:02:15.259142 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:15.259180 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:15.259712 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:15.259982 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:15.259986 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:15.260592 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:15.260803 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:15.260809 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.261546 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.261547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:15.262372 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:15.262375 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.262842 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.262843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:15.262978 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:15.263459 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:02:15.263467 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:15.264931 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:15.266550 (+ 1619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:15.266557 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:15.266559 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:15.266786 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:15.266983 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:15.266984 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:15.266984 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:15.267080 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:15.267332 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:15.267337 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:15.267451 (+ 114us) client_negotiation.cc:769] Sending connection context
0806 15:02:15.267536 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 15:02:15.267661 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":83,"threads_started":1}
I20260806 15:02:15.268051 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 4738
I20260806 15:02:15.268157 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance
I20260806 15:02:15.268433 11297 external_mini_cluster.cc:1428] Setting key 2c444a63a7b015a6094d786f785a3140
I20260806 15:02:15.268522 4906 heartbeater.cc:347] Connected to a master server at 127.11.8.126:40051
I20260806 15:02:15.268638 4906 heartbeater.cc:464] Registering TS with master...
I20260806 15:02:15.268888 4906 heartbeater.cc:511] Master 127.11.8.126:40051 requested a full tablet report, sending...
I20260806 15:02:15.269320 4274 ts_manager.cc:194] Registered new tserver with Master: de3141875c6849df93e652c86b5196a5 (127.11.8.67:39459)
I20260806 15:02:15.269738 4274 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.67@KRBTEST.COM'} at 127.11.8.67:42007
I20260806 15:02:15.280754 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:02:15.289202 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:15.282401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40356 (local address 127.11.8.126:40051)
0806 15:02:15.282530 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:15.282532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:15.282605 (+ 73us) server_negotiation.cc:407] Connection header received
0806 15:02:15.282681 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:15.282682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:15.282693 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:15.282773 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:15.283445 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.283949 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.284701 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.284898 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.284977 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:15.286551 (+ 1574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:15.286553 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:15.286553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:15.286561 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:15.287808 (+ 1247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:15.288298 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:15.288300 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:15.288300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:15.288361 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:15.288594 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:15.288595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:15.288595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:15.288747 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:15.288868 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:15.289022 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 15:02:15.289088 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 15:02:15.291525 4274 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:02:15.293207 4274 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:02:15.300047 4934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:15.296199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59796 (local address 127.11.8.65:34627)
0806 15:02:15.296378 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:15.296383 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:15.296392 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:02:15.296430 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:15.296432 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:15.296448 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:15.296677 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:15.296746 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.297786 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.298943 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.299651 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.299731 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:15.299789 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:15.299925 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 15:02:15.299966 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":94,"threads_started":1}
I20260806 15:02:15.300066 4933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:15.296059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:34627 (local address 127.0.0.1:59796)
0806 15:02:15.296185 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:15.296194 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:15.296274 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:15.296550 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:15.296551 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:15.296554 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:15.296598 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:15.296600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.297903 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.297904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:15.298858 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:15.298861 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.299750 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.299751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:15.299806 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:15.299817 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 15:02:15.299908 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 15:02:15.299961 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":1,"thread_start_us":73,"threads_started":1}
I20260806 15:02:15.300169 4929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:15.295853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:39459 (local address 127.0.0.1:41554)
0806 15:02:15.296165 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:15.296175 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:15.296222 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:15.296685 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:15.296686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:15.296689 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:15.296737 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:15.296739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.297661 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.297662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:15.299038 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:15.299040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.299919 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.299920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:15.299971 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:15.299974 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:02:15.300027 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 15:02:15.300060 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":6,"thread_start_us":89,"threads_started":1}
I20260806 15:02:15.300370 4932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:15.295963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41554 (local address 127.11.8.67:39459)
0806 15:02:15.296503 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:15.296508 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:15.296519 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:02:15.296561 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:15.296564 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:15.296581 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:15.296819 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:15.296899 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.297553 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.299130 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.299832 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.299900 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:15.299951 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:15.300251 (+ 300us) server_negotiation.cc:299] Negotiation successful
0806 15:02:15.300303 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":98,"threads_started":1}
I20260806 15:02:15.301131 4925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:15.295627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:40027 (local address 127.0.0.1:44302)
0806 15:02:15.295769 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:15.295809 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:15.295872 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:15.296284 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:15.296288 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:15.296290 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:15.296376 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:15.296378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.297833 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.297834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:15.299037 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:15.299040 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.300893 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.300894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:15.300960 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:15.300963 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:02:15.301012 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 15:02:15.301053 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":31,"thread_start_us":77,"threads_started":1}
I20260806 15:02:15.301179 4824 tablet_service.cc:1467] Processing CreateTablet for tablet 57eb50fd5dbe449d9a8382d8b8f00cc2 (DEFAULT_TABLE table=test-table [id=f70b35f92e7d48a4bf295015aa9f6172]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:02:15.301399 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:15.295754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44302 (local address 127.11.8.66:40027)
0806 15:02:15.296015 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:15.296022 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:15.296034 (+ 12us) server_negotiation.cc:407] Connection header received
0806 15:02:15.296077 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:15.296080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:15.296095 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:15.296190 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:15.296584 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.297726 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.299989 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.300730 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.300803 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:15.300859 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:15.301263 (+ 404us) server_negotiation.cc:299] Negotiation successful
0806 15:02:15.301307 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":101,"threads_started":1}
I20260806 15:02:15.301594 4824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57eb50fd5dbe449d9a8382d8b8f00cc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:15.301983 4644 tablet_service.cc:1467] Processing CreateTablet for tablet 57eb50fd5dbe449d9a8382d8b8f00cc2 (DEFAULT_TABLE table=test-table [id=f70b35f92e7d48a4bf295015aa9f6172]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:02:15.302294 4644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57eb50fd5dbe449d9a8382d8b8f00cc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:15.304477 4935 tablet_bootstrap.cc:492] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P de3141875c6849df93e652c86b5196a5: Bootstrap starting.
I20260806 15:02:15.304819 4446 tablet_service.cc:1467] Processing CreateTablet for tablet 57eb50fd5dbe449d9a8382d8b8f00cc2 (DEFAULT_TABLE table=test-table [id=f70b35f92e7d48a4bf295015aa9f6172]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:02:15.305157 4446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57eb50fd5dbe449d9a8382d8b8f00cc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:15.305274 4935 tablet_bootstrap.cc:654] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P de3141875c6849df93e652c86b5196a5: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:15.305197 4937 tablet_bootstrap.cc:492] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P 3d05bb57b47d45e3a4206536abc58f89: Bootstrap starting.
I20260806 15:02:15.305769 4935 log.cc:824] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P de3141875c6849df93e652c86b5196a5: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:15.305892 4937 tablet_bootstrap.cc:654] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P 3d05bb57b47d45e3a4206536abc58f89: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:15.306492 4937 log.cc:824] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P 3d05bb57b47d45e3a4206536abc58f89: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:15.306540 4935 tablet_bootstrap.cc:492] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P de3141875c6849df93e652c86b5196a5: No bootstrap required, opened a new log
I20260806 15:02:15.306627 4935 ts_tablet_manager.cc:1397] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P de3141875c6849df93e652c86b5196a5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:02:15.307273 4937 tablet_bootstrap.cc:492] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P 3d05bb57b47d45e3a4206536abc58f89: No bootstrap required, opened a new log
I20260806 15:02:15.307339 4937 ts_tablet_manager.cc:1397] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P 3d05bb57b47d45e3a4206536abc58f89: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:02:15.308308 4935 raft_consensus.cc:348] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P de3141875c6849df93e652c86b5196a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d05bb57b47d45e3a4206536abc58f89" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 40027 } } peers { permanent_uuid: "de3141875c6849df93e652c86b5196a5" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 39459 } } peers { permanent_uuid: "aa377ec71efd49b19cfd7ecd17e3686b" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34627 } }
I20260806 15:02:15.308473 4935 raft_consensus.cc:374] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P de3141875c6849df93e652c86b5196a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:15.308523 4935 raft_consensus.cc:729] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P de3141875c6849df93e652c86b5196a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de3141875c6849df93e652c86b5196a5, State: Initialized, Role: FOLLOWER
I20260806 15:02:15.308637 4935 consensus_queue.cc:260] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P de3141875c6849df93e652c86b5196a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d05bb57b47d45e3a4206536abc58f89" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 40027 } } peers { permanent_uuid: "de3141875c6849df93e652c86b5196a5" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 39459 } } peers { permanent_uuid: "aa377ec71efd49b19cfd7ecd17e3686b" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34627 } }
I20260806 15:02:15.308821 4944 tablet_bootstrap.cc:492] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P aa377ec71efd49b19cfd7ecd17e3686b: Bootstrap starting.
I20260806 15:02:15.308933 4935 ts_tablet_manager.cc:1428] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P de3141875c6849df93e652c86b5196a5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:02:15.309053 4906 heartbeater.cc:503] Master 127.11.8.126:40051 was elected leader, sending a full tablet report...
I20260806 15:02:15.309401 4937 raft_consensus.cc:348] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P 3d05bb57b47d45e3a4206536abc58f89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d05bb57b47d45e3a4206536abc58f89" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 40027 } } peers { permanent_uuid: "de3141875c6849df93e652c86b5196a5" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 39459 } } peers { permanent_uuid: "aa377ec71efd49b19cfd7ecd17e3686b" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34627 } }
I20260806 15:02:15.309576 4937 raft_consensus.cc:374] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P 3d05bb57b47d45e3a4206536abc58f89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:15.309620 4937 raft_consensus.cc:729] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P 3d05bb57b47d45e3a4206536abc58f89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d05bb57b47d45e3a4206536abc58f89, State: Initialized, Role: FOLLOWER
I20260806 15:02:15.309746 4937 consensus_queue.cc:260] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P 3d05bb57b47d45e3a4206536abc58f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d05bb57b47d45e3a4206536abc58f89" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 40027 } } peers { permanent_uuid: "de3141875c6849df93e652c86b5196a5" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 39459 } } peers { permanent_uuid: "aa377ec71efd49b19cfd7ecd17e3686b" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34627 } }
I20260806 15:02:15.309784 4944 tablet_bootstrap.cc:654] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P aa377ec71efd49b19cfd7ecd17e3686b: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:15.310043 4937 ts_tablet_manager.cc:1428] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P 3d05bb57b47d45e3a4206536abc58f89: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 15:02:15.310092 4730 heartbeater.cc:503] Master 127.11.8.126:40051 was elected leader, sending a full tablet report...
I20260806 15:02:15.310364 4944 log.cc:824] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P aa377ec71efd49b19cfd7ecd17e3686b: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:15.311267 4944 tablet_bootstrap.cc:492] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P aa377ec71efd49b19cfd7ecd17e3686b: No bootstrap required, opened a new log
I20260806 15:02:15.311323 4944 ts_tablet_manager.cc:1397] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P aa377ec71efd49b19cfd7ecd17e3686b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 15:02:15.312588 4944 raft_consensus.cc:348] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P aa377ec71efd49b19cfd7ecd17e3686b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d05bb57b47d45e3a4206536abc58f89" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 40027 } } peers { permanent_uuid: "de3141875c6849df93e652c86b5196a5" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 39459 } } peers { permanent_uuid: "aa377ec71efd49b19cfd7ecd17e3686b" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34627 } }
I20260806 15:02:15.312709 4944 raft_consensus.cc:374] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P aa377ec71efd49b19cfd7ecd17e3686b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:15.312731 4944 raft_consensus.cc:729] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P aa377ec71efd49b19cfd7ecd17e3686b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa377ec71efd49b19cfd7ecd17e3686b, State: Initialized, Role: FOLLOWER
I20260806 15:02:15.312844 4944 consensus_queue.cc:260] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P aa377ec71efd49b19cfd7ecd17e3686b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d05bb57b47d45e3a4206536abc58f89" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 40027 } } peers { permanent_uuid: "de3141875c6849df93e652c86b5196a5" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 39459 } } peers { permanent_uuid: "aa377ec71efd49b19cfd7ecd17e3686b" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34627 } }
I20260806 15:02:15.313136 4944 ts_tablet_manager.cc:1428] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P aa377ec71efd49b19cfd7ecd17e3686b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:02:15.313146 4546 heartbeater.cc:503] Master 127.11.8.126:40051 was elected leader, sending a full tablet report...
W20260806 15:02:15.331213 4731 tablet.cc:2369] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P 3d05bb57b47d45e3a4206536abc58f89: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 15:02:15.384425 4547 tablet.cc:2369] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P aa377ec71efd49b19cfd7ecd17e3686b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:02:15.392409 4948 raft_consensus.cc:482] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P aa377ec71efd49b19cfd7ecd17e3686b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:02:15.392520 4948 raft_consensus.cc:504] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P aa377ec71efd49b19cfd7ecd17e3686b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d05bb57b47d45e3a4206536abc58f89" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 40027 } } peers { permanent_uuid: "de3141875c6849df93e652c86b5196a5" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 39459 } } peers { permanent_uuid: "aa377ec71efd49b19cfd7ecd17e3686b" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34627 } }
I20260806 15:02:15.392848 4948 leader_election.cc:288] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P aa377ec71efd49b19cfd7ecd17e3686b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d05bb57b47d45e3a4206536abc58f89 (127.11.8.66:40027), de3141875c6849df93e652c86b5196a5 (127.11.8.67:39459)
I20260806 15:02:15.396839 4949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:15.393185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:39459 (local address 127.11.8.65:39843)
0806 15:02:15.393292 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:15.393300 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:15.393369 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:15.393485 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:15.393486 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:15.393487 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:15.393549 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:15.393550 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.394381 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.394382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:15.395732 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:15.395735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.396538 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.396539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:15.396616 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:15.396621 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 15:02:15.396692 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 15:02:15.396740 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":48,"thread_start_us":63,"threads_started":1}
I20260806 15:02:15.396833 4932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:15.393241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:39843 (local address 127.11.8.67:39459)
0806 15:02:15.393306 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:15.393308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:15.393360 (+ 52us) server_negotiation.cc:407] Connection header received
0806 15:02:15.393419 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:15.393420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:15.393431 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:15.393480 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:15.393611 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.394298 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.395839 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.396447 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.396529 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:15.396541 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:15.396698 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 15:02:15.396762 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 15:02:15.396924 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:15.393176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:41039 (local address 127.11.8.66:40027)
0806 15:02:15.393233 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:15.393235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:15.393246 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:02:15.393278 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:15.393279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:15.393292 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:15.393358 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:15.393530 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.394288 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.395839 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.396452 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.396530 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:15.396542 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:15.396796 (+ 254us) server_negotiation.cc:299] Negotiation successful
0806 15:02:15.396850 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 15:02:15.397064 4548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:15.393035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:40027 (local address 127.11.8.65:41039)
0806 15:02:15.393134 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:15.393147 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:15.393208 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:15.393374 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:15.393375 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:15.393377 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:15.393450 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:15.393452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.394381 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.394382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:15.395732 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:15.395735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.396538 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.396539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:15.396658 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:15.396660 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 15:02:15.396710 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 15:02:15.396747 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"mutex_wait_us":36}
I20260806 15:02:15.397327 4850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57eb50fd5dbe449d9a8382d8b8f00cc2" candidate_uuid: "aa377ec71efd49b19cfd7ecd17e3686b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de3141875c6849df93e652c86b5196a5" is_pre_election: true
I20260806 15:02:15.397518 4850 raft_consensus.cc:2454] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P de3141875c6849df93e652c86b5196a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa377ec71efd49b19cfd7ecd17e3686b in term 0.
I20260806 15:02:15.397508 4670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57eb50fd5dbe449d9a8382d8b8f00cc2" candidate_uuid: "aa377ec71efd49b19cfd7ecd17e3686b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d05bb57b47d45e3a4206536abc58f89" is_pre_election: true
I20260806 15:02:15.397686 4670 raft_consensus.cc:2454] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P 3d05bb57b47d45e3a4206536abc58f89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa377ec71efd49b19cfd7ecd17e3686b in term 0.
I20260806 15:02:15.397774 4383 leader_election.cc:302] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P aa377ec71efd49b19cfd7ecd17e3686b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa377ec71efd49b19cfd7ecd17e3686b, de3141875c6849df93e652c86b5196a5; no voters:
I20260806 15:02:15.397921 4948 raft_consensus.cc:2781] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P aa377ec71efd49b19cfd7ecd17e3686b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:02:15.397982 4948 raft_consensus.cc:482] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P aa377ec71efd49b19cfd7ecd17e3686b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:02:15.398020 4948 raft_consensus.cc:3037] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P aa377ec71efd49b19cfd7ecd17e3686b [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:15.398850 4948 raft_consensus.cc:504] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P aa377ec71efd49b19cfd7ecd17e3686b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d05bb57b47d45e3a4206536abc58f89" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 40027 } } peers { permanent_uuid: "de3141875c6849df93e652c86b5196a5" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 39459 } } peers { permanent_uuid: "aa377ec71efd49b19cfd7ecd17e3686b" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34627 } }
I20260806 15:02:15.398993 4948 leader_election.cc:288] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P aa377ec71efd49b19cfd7ecd17e3686b [CANDIDATE]: Term 1 election: Requested vote from peers 3d05bb57b47d45e3a4206536abc58f89 (127.11.8.66:40027), de3141875c6849df93e652c86b5196a5 (127.11.8.67:39459)
I20260806 15:02:15.399165 4850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57eb50fd5dbe449d9a8382d8b8f00cc2" candidate_uuid: "aa377ec71efd49b19cfd7ecd17e3686b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de3141875c6849df93e652c86b5196a5"
I20260806 15:02:15.399188 4670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57eb50fd5dbe449d9a8382d8b8f00cc2" candidate_uuid: "aa377ec71efd49b19cfd7ecd17e3686b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d05bb57b47d45e3a4206536abc58f89"
I20260806 15:02:15.399266 4850 raft_consensus.cc:3037] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P de3141875c6849df93e652c86b5196a5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:15.399260 4670 raft_consensus.cc:3037] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P 3d05bb57b47d45e3a4206536abc58f89 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:15.399979 4670 raft_consensus.cc:2454] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P 3d05bb57b47d45e3a4206536abc58f89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa377ec71efd49b19cfd7ecd17e3686b in term 1.
I20260806 15:02:15.399991 4850 raft_consensus.cc:2454] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P de3141875c6849df93e652c86b5196a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa377ec71efd49b19cfd7ecd17e3686b in term 1.
I20260806 15:02:15.400142 4383 leader_election.cc:302] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P aa377ec71efd49b19cfd7ecd17e3686b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa377ec71efd49b19cfd7ecd17e3686b, de3141875c6849df93e652c86b5196a5; no voters:
I20260806 15:02:15.400264 4948 raft_consensus.cc:2781] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P aa377ec71efd49b19cfd7ecd17e3686b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:15.400425 4948 raft_consensus.cc:686] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P aa377ec71efd49b19cfd7ecd17e3686b [term 1 LEADER]: Becoming Leader. State: Replica: aa377ec71efd49b19cfd7ecd17e3686b, State: Running, Role: LEADER
I20260806 15:02:15.400534 4948 consensus_queue.cc:237] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P aa377ec71efd49b19cfd7ecd17e3686b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d05bb57b47d45e3a4206536abc58f89" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 40027 } } peers { permanent_uuid: "de3141875c6849df93e652c86b5196a5" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 39459 } } peers { permanent_uuid: "aa377ec71efd49b19cfd7ecd17e3686b" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34627 } }
I20260806 15:02:15.401376 4274 catalog_manager.cc:5495] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P aa377ec71efd49b19cfd7ecd17e3686b reported cstate change: term changed from 0 to 1, leader changed from <none> to aa377ec71efd49b19cfd7ecd17e3686b (127.11.8.65). New cstate: current_term: 1 leader_uuid: "aa377ec71efd49b19cfd7ecd17e3686b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d05bb57b47d45e3a4206536abc58f89" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 40027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de3141875c6849df93e652c86b5196a5" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 39459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa377ec71efd49b19cfd7ecd17e3686b" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 34627 } health_report { overall_health: HEALTHY } } }
I20260806 15:02:15.416778 4934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:15.414388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59800 (local address 127.11.8.65:34627)
0806 15:02:15.414470 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:15.414472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:15.414484 (+ 12us) server_negotiation.cc:407] Connection header received
0806 15:02:15.414549 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:15.414550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:15.414569 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:15.414633 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:02:15.414757 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.415216 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.416010 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.416188 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.416254 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:15.416459 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:02:15.416546 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:02:15.416621 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:15.416680 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 15:02:15.416688 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 15:02:15.418751 4670 raft_consensus.cc:1261] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P 3d05bb57b47d45e3a4206536abc58f89 [term 1 FOLLOWER]: Refusing update from remote peer aa377ec71efd49b19cfd7ecd17e3686b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:02:15.418751 4850 raft_consensus.cc:1261] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P de3141875c6849df93e652c86b5196a5 [term 1 FOLLOWER]: Refusing update from remote peer aa377ec71efd49b19cfd7ecd17e3686b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:02:15.419152 4948 consensus_queue.cc:1035] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P aa377ec71efd49b19cfd7ecd17e3686b [LEADER]: Connected to new peer: Peer: permanent_uuid: "de3141875c6849df93e652c86b5196a5" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 39459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:02:15.419322 4950 consensus_queue.cc:1035] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P aa377ec71efd49b19cfd7ecd17e3686b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d05bb57b47d45e3a4206536abc58f89" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 40027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:02:15.421736 4953 mvcc.cc:205] Tried to move back new op lower bound from 7315572881071792128 to 7315572881001345024. Current Snapshot: MvccSnapshot[applied={T|T < 7315572881071792128}]
I20260806 15:02:15.422055 4954 mvcc.cc:205] Tried to move back new op lower bound from 7315572881071792128 to 7315572881001345024. Current Snapshot: MvccSnapshot[applied={T|T < 7315572881071792128}]
I20260806 15:02:15.422526 4955 mvcc.cc:205] Tried to move back new op lower bound from 7315572881071792128 to 7315572881001345024. Current Snapshot: MvccSnapshot[applied={T|T < 7315572881071792128}]
I20260806 15:02:15.424860 4274 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:02:15.424965 4274 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:02:15.425753 4274 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5b53028b1cb34f68a50ab2cf8c507c87: Sending DeleteTablet for 3 replicas of tablet 57eb50fd5dbe449d9a8382d8b8f00cc2
I20260806 15:02:15.426172 4644 tablet_service.cc:1514] Processing DeleteTablet for tablet 57eb50fd5dbe449d9a8382d8b8f00cc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:02:15 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:44302
I20260806 15:02:15.426376 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 4361
I20260806 15:02:15.426524 4824 tablet_service.cc:1514] Processing DeleteTablet for tablet 57eb50fd5dbe449d9a8382d8b8f00cc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:02:15 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:41554
I20260806 15:02:15.431823 4967 tablet_replica.cc:330] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P de3141875c6849df93e652c86b5196a5: stopping tablet replica
I20260806 15:02:15.431970 4967 raft_consensus.cc:2229] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P de3141875c6849df93e652c86b5196a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:02:15.432116 4967 raft_consensus.cc:2258] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P de3141875c6849df93e652c86b5196a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:02:15.432317 4965 tablet_replica.cc:330] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P 3d05bb57b47d45e3a4206536abc58f89: stopping tablet replica
I20260806 15:02:15.432411 4965 raft_consensus.cc:2229] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P 3d05bb57b47d45e3a4206536abc58f89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:02:15.432494 4965 raft_consensus.cc:2258] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P 3d05bb57b47d45e3a4206536abc58f89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:02:15.433527 4965 ts_tablet_manager.cc:1905] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P 3d05bb57b47d45e3a4206536abc58f89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:02:15.435348 4967 ts_tablet_manager.cc:1905] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P de3141875c6849df93e652c86b5196a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:02:15.435748 4965 ts_tablet_manager.cc:1918] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P 3d05bb57b47d45e3a4206536abc58f89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:02:15.435844 4965 log.cc:1196] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P 3d05bb57b47d45e3a4206536abc58f89: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-1/wal/wals/57eb50fd5dbe449d9a8382d8b8f00cc2
I20260806 15:02:15.436153 4965 ts_tablet_manager.cc:1939] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P 3d05bb57b47d45e3a4206536abc58f89: Deleting consensus metadata
I20260806 15:02:15.436617 4252 catalog_manager.cc:4841] TS 3d05bb57b47d45e3a4206536abc58f89 (127.11.8.66:40027): tablet 57eb50fd5dbe449d9a8382d8b8f00cc2 (table test-table [id=f70b35f92e7d48a4bf295015aa9f6172]) successfully deleted
I20260806 15:02:15.437516 4967 ts_tablet_manager.cc:1918] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P de3141875c6849df93e652c86b5196a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:02:15.437619 4967 log.cc:1196] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P de3141875c6849df93e652c86b5196a5: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786028287744255-11297-0/minicluster-data/ts-2/wal/wals/57eb50fd5dbe449d9a8382d8b8f00cc2
I20260806 15:02:15.437927 4967 ts_tablet_manager.cc:1939] T 57eb50fd5dbe449d9a8382d8b8f00cc2 P de3141875c6849df93e652c86b5196a5: Deleting consensus metadata
I20260806 15:02:15.438365 4252 catalog_manager.cc:4841] TS de3141875c6849df93e652c86b5196a5 (127.11.8.67:39459): tablet 57eb50fd5dbe449d9a8382d8b8f00cc2 (table test-table [id=f70b35f92e7d48a4bf295015aa9f6172]) successfully deleted
W20260806 15:02:15.442117 4252 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.65:34627 (error 108)
I20260806 15:02:15.442346 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 4553
I20260806 15:02:15.444167 4925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:15.443997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:34627 (local address 127.0.0.1:59812)
0806 15:02:15.444067 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:15.444124 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.65:34627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 15:02:15.445776 4252 catalog_manager.cc:4584] TS aa377ec71efd49b19cfd7ecd17e3686b (127.11.8.65:34627): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 57eb50fd5dbe449d9a8382d8b8f00cc2: Network error: Client connection negotiation failed: client connection to 127.11.8.65:34627: connect: Connection refused (error 111)
I20260806 15:02:15.450522 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 4738
I20260806 15:02:15.461050 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 4226
2026-08-06T15:02:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2291 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T15:02:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:02:15Z Disabled control of system clock
I20260806 15:02:15.494019 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:34073
--webserver_interface=127.11.8.126
--webserver_port=0
--builtin_ntp_servers=127.11.8.84:41285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:34073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 15:02:15.574986 4973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:15.575196 4973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:15.575217 4973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:15.576365 4973 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:15.576407 4973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:15.576421 4973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:15.576434 4973 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:15.576447 4973 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:15.577785 4973 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:41285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:34073
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:34073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:15.578040 4973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:15.578334 4973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:15.583099 4983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:15.583144 4981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:15.583094 4980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:15.583389 4973 server_base.cc:1034] running on GCE node
I20260806 15:02:15.583637 4973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:15.583874 4973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:15.585027 4973 hybrid_clock.cc:648] HybridClock initialized: now 1786028535585007 us; error 33 us; skew 500 ppm
I20260806 15:02:15.586234 4973 webserver.cc:460] Webserver started at http://127.11.8.126:35087/ using document root <none> and password file <none>
I20260806 15:02:15.586556 4973 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:15.586607 4973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:15.586764 4973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:15.587802 4973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "ed02150beea94838a1216beac7966226"
format_stamp: "Formatted at 2026-08-06 15:02:15 on dist-test-slave-68xk"
server_key: "dca4637c1546f776b00e129ad26028c2"
server_key_iv: "bb60f3b473ea8f0a06bfdc47a199f1b1"
server_key_version: "encryptionkey@0"
I20260806 15:02:15.588184 4973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "ed02150beea94838a1216beac7966226"
format_stamp: "Formatted at 2026-08-06 15:02:15 on dist-test-slave-68xk"
server_key: "dca4637c1546f776b00e129ad26028c2"
server_key_iv: "bb60f3b473ea8f0a06bfdc47a199f1b1"
server_key_version: "encryptionkey@0"
I20260806 15:02:15.589529 4973 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:02:15.590360 4991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:15.590574 4973 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:02:15.590667 4973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "ed02150beea94838a1216beac7966226"
format_stamp: "Formatted at 2026-08-06 15:02:15 on dist-test-slave-68xk"
server_key: "dca4637c1546f776b00e129ad26028c2"
server_key_iv: "bb60f3b473ea8f0a06bfdc47a199f1b1"
server_key_version: "encryptionkey@0"
I20260806 15:02:15.590755 4973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:15.597543 4973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:15.597847 4973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:15.597991 4973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:15.602154 4973 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:34073
I20260806 15:02:15.602221 5064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:34073 every 8 connection(s)
I20260806 15:02:15.602626 4973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:02:15.603607 5065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:15.605955 5065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed02150beea94838a1216beac7966226: Bootstrap starting.
I20260806 15:02:15.606618 5065 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed02150beea94838a1216beac7966226: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:15.606936 5065 log.cc:824] T 00000000000000000000000000000000 P ed02150beea94838a1216beac7966226: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:15.607613 5065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed02150beea94838a1216beac7966226: No bootstrap required, opened a new log
I20260806 15:02:15.608633 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 4973
I20260806 15:02:15.608784 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:02:15.609025 11297 external_mini_cluster.cc:1428] Setting key f68e49563f6cdd5c9a2438b0f84a02e8
I20260806 15:02:15.609927 5065 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed02150beea94838a1216beac7966226 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed02150beea94838a1216beac7966226" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34073 } }
I20260806 15:02:15.610067 5065 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed02150beea94838a1216beac7966226 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:15.610090 5065 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed02150beea94838a1216beac7966226 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed02150beea94838a1216beac7966226, State: Initialized, Role: FOLLOWER
I20260806 15:02:15.610222 5065 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed02150beea94838a1216beac7966226 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed02150beea94838a1216beac7966226" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34073 } }
I20260806 15:02:15.610314 5065 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed02150beea94838a1216beac7966226 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:02:15.610347 5065 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed02150beea94838a1216beac7966226 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:02:15.610400 5065 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed02150beea94838a1216beac7966226 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:15.610978 5065 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed02150beea94838a1216beac7966226 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed02150beea94838a1216beac7966226" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34073 } }
I20260806 15:02:15.611068 5065 leader_election.cc:302] T 00000000000000000000000000000000 P ed02150beea94838a1216beac7966226 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed02150beea94838a1216beac7966226; no voters:
I20260806 15:02:15.611266 5065 leader_election.cc:288] T 00000000000000000000000000000000 P ed02150beea94838a1216beac7966226 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:02:15.611330 5072 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed02150beea94838a1216beac7966226 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:15.611469 5072 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed02150beea94838a1216beac7966226 [term 1 LEADER]: Becoming Leader. State: Replica: ed02150beea94838a1216beac7966226, State: Running, Role: LEADER
I20260806 15:02:15.611567 5065 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed02150beea94838a1216beac7966226 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:15.611558 5072 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed02150beea94838a1216beac7966226 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed02150beea94838a1216beac7966226" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34073 } }
I20260806 15:02:15.611876 5072 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed02150beea94838a1216beac7966226 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed02150beea94838a1216beac7966226. Latest consensus state: current_term: 1 leader_uuid: "ed02150beea94838a1216beac7966226" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed02150beea94838a1216beac7966226" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34073 } } }
I20260806 15:02:15.611958 5072 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed02150beea94838a1216beac7966226 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:15.611891 5074 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed02150beea94838a1216beac7966226 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed02150beea94838a1216beac7966226" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed02150beea94838a1216beac7966226" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34073 } } }
I20260806 15:02:15.612139 5079 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:02:15.612183 5074 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed02150beea94838a1216beac7966226 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:15.612697 5079 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:02:15.614526 5079 catalog_manager.cc:1347] Generated new cluster ID: 6ec18634ad2e4efd9fab199b4bdd6f9d
I20260806 15:02:15.614579 5079 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:02:15.615769 5070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:15.609618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45748 (local address 127.11.8.126:34073)
0806 15:02:15.609843 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:15.609851 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:15.609880 (+ 29us) server_negotiation.cc:407] Connection header received
0806 15:02:15.609971 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:15.609976 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:15.610027 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:15.610185 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:15.610536 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.612231 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.613464 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.614019 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.614169 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:15.615027 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:15.615040 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:15.615041 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:15.615259 (+ 218us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:15.615307 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:15.615322 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:15.615399 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:15.615470 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 15:02:15.615555 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":104,"threads_started":1}
I20260806 15:02:15.625368 5079 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:02:15.625854 5079 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:02:15.635782 5079 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed02150beea94838a1216beac7966226: Generated new TSK 0
I20260806 15:02:15.635960 5079 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:02:15.640846 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:0
--local_ip_for_outbound_sockets=127.11.8.65
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:34073
--builtin_ntp_servers=127.11.8.84:41285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 15:02:15.727985 5094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:15.728242 5094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:15.728281 5094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:15.729436 5094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:15.729503 5094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:15.729544 5094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 15:02:15.730985 5094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:41285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:34073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:15.731266 5094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:15.731568 5094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:15.737439 5100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:15.737560 5101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:15.737605 5103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:15.737808 5094 server_base.cc:1034] running on GCE node
I20260806 15:02:15.738018 5094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:15.738276 5094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:15.739444 5094 hybrid_clock.cc:648] HybridClock initialized: now 1786028535739429 us; error 33 us; skew 500 ppm
I20260806 15:02:15.740744 5094 webserver.cc:460] Webserver started at http://127.11.8.65:38311/ using document root <none> and password file <none>
I20260806 15:02:15.741011 5094 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:15.741084 5094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:15.741240 5094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:15.742460 5094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-0/data/instance:
uuid: "48ad566658aa4bcb89c96a8a39b116df"
format_stamp: "Formatted at 2026-08-06 15:02:15 on dist-test-slave-68xk"
server_key: "330d6ee9615bf159a8061e87f41ad702"
server_key_iv: "f7226348315926df288de37fad43bbe2"
server_key_version: "encryptionkey@0"
I20260806 15:02:15.742861 5094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance:
uuid: "48ad566658aa4bcb89c96a8a39b116df"
format_stamp: "Formatted at 2026-08-06 15:02:15 on dist-test-slave-68xk"
server_key: "330d6ee9615bf159a8061e87f41ad702"
server_key_iv: "f7226348315926df288de37fad43bbe2"
server_key_version: "encryptionkey@0"
I20260806 15:02:15.744217 5094 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:02:15.744959 5111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:15.745153 5094 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:02:15.745270 5094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "48ad566658aa4bcb89c96a8a39b116df"
format_stamp: "Formatted at 2026-08-06 15:02:15 on dist-test-slave-68xk"
server_key: "330d6ee9615bf159a8061e87f41ad702"
server_key_iv: "f7226348315926df288de37fad43bbe2"
server_key_version: "encryptionkey@0"
I20260806 15:02:15.745347 5094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:15.754783 5094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:15.755067 5094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:15.755218 5094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:15.755445 5094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:02:15.755777 5094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:02:15.755833 5094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:15.755868 5094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:02:15.755896 5094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:15.761544 5094 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:46269
I20260806 15:02:15.761648 5261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:46269 every 8 connection(s)
I20260806 15:02:15.761948 5094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
I20260806 15:02:15.767340 5070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:15.762781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:52673 (local address 127.11.8.126:34073)
0806 15:02:15.762878 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:15.762881 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:15.763057 (+ 176us) server_negotiation.cc:407] Connection header received
0806 15:02:15.763550 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:15.763551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:15.763563 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:15.763618 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:15.764306 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.764791 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.765878 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.766065 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.766118 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:15.767116 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:15.767117 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:15.767118 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:15.767122 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:15.767135 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:15.767139 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:15.767180 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:15.767255 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 15:02:15.767281 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 15:02:15.767553 5265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:15.762705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34073 (local address 127.11.8.65:52673)
0806 15:02:15.762920 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:15.762957 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:15.763461 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:15.763676 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:15.763679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:15.763688 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:15.764229 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:15.764233 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.764879 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.764880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:15.765790 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:15.765793 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.766201 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.766203 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:15.766321 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:15.766984 (+ 663us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:02:15.766985 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:15.767013 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:02:15.767037 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:15.767183 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:15.767185 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:15.767187 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 15:02:15.767235 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 15:02:15.767314 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":73,"threads_started":1}
I20260806 15:02:15.768141 5262 heartbeater.cc:347] Connected to a master server at 127.11.8.126:34073
I20260806 15:02:15.768254 5262 heartbeater.cc:464] Registering TS with master...
I20260806 15:02:15.768438 5262 heartbeater.cc:511] Master 127.11.8.126:34073 requested a full tablet report, sending...
I20260806 15:02:15.769033 5013 ts_manager.cc:194] Registered new tserver with Master: 48ad566658aa4bcb89c96a8a39b116df (127.11.8.65:46269)
I20260806 15:02:15.769749 5013 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.8.65:52673
I20260806 15:02:15.769857 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 5094
I20260806 15:02:15.769953 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance
I20260806 15:02:15.770210 11297 external_mini_cluster.cc:1428] Setting key 192744c34b71db73822c34adde30fd28
I20260806 15:02:15.771322 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:0
--local_ip_for_outbound_sockets=127.11.8.66
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:34073
--builtin_ntp_servers=127.11.8.84:41285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 15:02:15.856935 5266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:15.857146 5266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:15.857168 5266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:15.858258 5266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:15.858292 5266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:15.858316 5266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 15:02:15.859700 5266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:41285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:34073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:15.859884 5266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:15.860122 5266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:15.865123 5274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:15.865188 5281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:15.865224 5266 server_base.cc:1034] running on GCE node
W20260806 15:02:15.865334 5278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:15.865538 5266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:15.865821 5266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:15.866961 5266 hybrid_clock.cc:648] HybridClock initialized: now 1786028535866941 us; error 45 us; skew 500 ppm
I20260806 15:02:15.868173 5266 webserver.cc:460] Webserver started at http://127.11.8.66:40187/ using document root <none> and password file <none>
I20260806 15:02:15.868440 5266 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:15.868518 5266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:15.868680 5266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:15.869738 5266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-1/data/instance:
uuid: "a0c9892845ea4715aa8e463c3c5ed025"
format_stamp: "Formatted at 2026-08-06 15:02:15 on dist-test-slave-68xk"
server_key: "4dddadce5b9dbc2fa669abbe6befca84"
server_key_iv: "fac9cdb57a5c276545363ac45e013838"
server_key_version: "encryptionkey@0"
I20260806 15:02:15.870105 5266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance:
uuid: "a0c9892845ea4715aa8e463c3c5ed025"
format_stamp: "Formatted at 2026-08-06 15:02:15 on dist-test-slave-68xk"
server_key: "4dddadce5b9dbc2fa669abbe6befca84"
server_key_iv: "fac9cdb57a5c276545363ac45e013838"
server_key_version: "encryptionkey@0"
I20260806 15:02:15.871500 5266 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:02:15.872287 5287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:15.872465 5266 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:02:15.872558 5266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "a0c9892845ea4715aa8e463c3c5ed025"
format_stamp: "Formatted at 2026-08-06 15:02:15 on dist-test-slave-68xk"
server_key: "4dddadce5b9dbc2fa669abbe6befca84"
server_key_iv: "fac9cdb57a5c276545363ac45e013838"
server_key_version: "encryptionkey@0"
I20260806 15:02:15.872637 5266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:15.892474 5266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:15.892829 5266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:15.892997 5266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:15.893311 5266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:02:15.893764 5266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:02:15.893821 5266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:15.893865 5266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:02:15.893893 5266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:15.900141 5266 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:39997
I20260806 15:02:15.900283 5436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:39997 every 8 connection(s)
I20260806 15:02:15.900571 5266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
I20260806 15:02:15.905992 5070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:15.901375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:56377 (local address 127.11.8.126:34073)
0806 15:02:15.901471 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:15.901473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:15.901692 (+ 219us) server_negotiation.cc:407] Connection header received
0806 15:02:15.902176 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:15.902177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:15.902188 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:15.902246 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:15.902986 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.903488 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.904486 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.904669 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.904721 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:15.905773 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:15.905774 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:15.905774 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:15.905778 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:15.905788 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:15.905792 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:15.905829 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:15.905909 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 15:02:15.905931 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 15:02:15.906236 5439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:15.901334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34073 (local address 127.11.8.66:56377)
0806 15:02:15.901556 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:15.901592 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:15.902083 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:15.902300 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:15.902304 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:15.902313 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:15.902907 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:15.902913 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.903587 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.903588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:15.904407 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:15.904409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:15.904772 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:15.904774 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:15.904920 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:15.905633 (+ 713us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:02:15.905634 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:15.905667 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:02:15.905695 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:15.905836 (+ 141us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:15.905839 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:15.905842 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:02:15.905896 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 15:02:15.905984 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":111,"threads_started":1}
I20260806 15:02:15.906788 5437 heartbeater.cc:347] Connected to a master server at 127.11.8.126:34073
I20260806 15:02:15.906878 5437 heartbeater.cc:464] Registering TS with master...
I20260806 15:02:15.907217 5437 heartbeater.cc:511] Master 127.11.8.126:34073 requested a full tablet report, sending...
I20260806 15:02:15.907440 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 5266
I20260806 15:02:15.907560 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance
I20260806 15:02:15.907670 5013 ts_manager.cc:194] Registered new tserver with Master: a0c9892845ea4715aa8e463c3c5ed025 (127.11.8.66:39997)
I20260806 15:02:15.907770 11297 external_mini_cluster.cc:1428] Setting key 67f787e471b796058c43819441c5e0ae
I20260806 15:02:15.908072 5013 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.8.66:56377
I20260806 15:02:15.908980 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:0
--local_ip_for_outbound_sockets=127.11.8.67
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:34073
--builtin_ntp_servers=127.11.8.84:41285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 15:02:15.995481 5440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:15.995698 5440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:15.995720 5440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:15.996820 5440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:15.996855 5440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:15.996879 5440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 15:02:15.998301 5440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:41285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:34073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:15.998559 5440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:15.998829 5440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:16.003755 5449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:16.003803 5446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:16.003832 5440 server_base.cc:1034] running on GCE node
W20260806 15:02:16.003756 5451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:16.004196 5440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:16.004498 5440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:16.005677 5440 hybrid_clock.cc:648] HybridClock initialized: now 1786028536005655 us; error 43 us; skew 500 ppm
I20260806 15:02:16.007177 5440 webserver.cc:460] Webserver started at http://127.11.8.67:43623/ using document root <none> and password file <none>
I20260806 15:02:16.007443 5440 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:16.007493 5440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:16.007642 5440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:16.008744 5440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-2/data/instance:
uuid: "98f4ab8149b846ffb7f63d0d9c7f2582"
format_stamp: "Formatted at 2026-08-06 15:02:16 on dist-test-slave-68xk"
server_key: "0b1e0a9a720b813c28a4147cb61d0404"
server_key_iv: "d5cb8d0f1a12ee05568fde64b0c7db2a"
server_key_version: "encryptionkey@0"
I20260806 15:02:16.009116 5440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance:
uuid: "98f4ab8149b846ffb7f63d0d9c7f2582"
format_stamp: "Formatted at 2026-08-06 15:02:16 on dist-test-slave-68xk"
server_key: "0b1e0a9a720b813c28a4147cb61d0404"
server_key_iv: "d5cb8d0f1a12ee05568fde64b0c7db2a"
server_key_version: "encryptionkey@0"
I20260806 15:02:16.010526 5440 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:02:16.011379 5457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:16.011574 5440 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:02:16.011672 5440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "98f4ab8149b846ffb7f63d0d9c7f2582"
format_stamp: "Formatted at 2026-08-06 15:02:16 on dist-test-slave-68xk"
server_key: "0b1e0a9a720b813c28a4147cb61d0404"
server_key_iv: "d5cb8d0f1a12ee05568fde64b0c7db2a"
server_key_version: "encryptionkey@0"
I20260806 15:02:16.011754 5440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:16.039767 5440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:16.040112 5440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:16.040275 5440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:16.040527 5440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:02:16.040894 5440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:02:16.040928 5440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:16.040952 5440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:02:16.040966 5440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:16.047204 5440 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:37581
I20260806 15:02:16.047298 5596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:37581 every 8 connection(s)
I20260806 15:02:16.047609 5440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
I20260806 15:02:16.052839 5070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:16.048402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:43187 (local address 127.11.8.126:34073)
0806 15:02:16.048459 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:16.048466 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:16.048748 (+ 282us) server_negotiation.cc:407] Connection header received
0806 15:02:16.049296 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:16.049298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:16.049308 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:16.049361 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:16.050042 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:16.050549 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:16.051531 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:16.051709 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:16.051768 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:16.052626 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:16.052628 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:16.052628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:16.052632 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:16.052645 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:16.052648 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:16.052691 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:16.052755 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 15:02:16.052780 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 15:02:16.053018 5599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:16.048420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:34073 (local address 127.11.8.67:43187)
0806 15:02:16.048617 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:16.048652 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:16.049200 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:16.049443 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:16.049447 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:16.049456 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:16.049971 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:16.049976 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:16.050632 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:16.050633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:16.051458 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:16.051461 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:16.051788 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:16.051789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:16.051899 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:16.052480 (+ 581us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:02:16.052481 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:16.052509 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:02:16.052532 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:16.052692 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:16.052694 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:16.052696 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 15:02:16.052742 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 15:02:16.052817 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":87,"threads_started":1}
I20260806 15:02:16.053573 5597 heartbeater.cc:347] Connected to a master server at 127.11.8.126:34073
I20260806 15:02:16.053678 5597 heartbeater.cc:464] Registering TS with master...
I20260806 15:02:16.053858 5597 heartbeater.cc:511] Master 127.11.8.126:34073 requested a full tablet report, sending...
I20260806 15:02:16.054270 5012 ts_manager.cc:194] Registered new tserver with Master: 98f4ab8149b846ffb7f63d0d9c7f2582 (127.11.8.67:37581)
I20260806 15:02:16.054714 5012 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.8.67:43187
I20260806 15:02:16.055512 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 5440
I20260806 15:02:16.055603 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance
I20260806 15:02:16.055763 11297 external_mini_cluster.cc:1428] Setting key 213420b05821ab16028e3e569c372e2e
I20260806 15:02:16.056641 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:02:16.060704 5070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:16.057992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45754 (local address 127.11.8.126:34073)
0806 15:02:16.058068 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:16.058070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:16.058135 (+ 65us) server_negotiation.cc:407] Connection header received
0806 15:02:16.058172 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:16.058172 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:16.058181 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:16.058226 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:16.058368 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:16.058854 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:16.059551 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:16.059715 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:16.059768 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:16.060437 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:16.060438 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:16.060439 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:16.060442 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:16.060451 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:16.060454 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:16.060510 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:16.060585 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 15:02:16.060615 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 15:02:16.062258 5012 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:02:16.063855 5012 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:02:16.070045 5612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:16.066313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:46269 (local address 127.0.0.1:54284)
0806 15:02:16.066504 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:16.066514 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:16.066557 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:16.066878 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:16.066879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:16.066882 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:16.066963 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:16.066965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:16.067968 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:16.067969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:16.069013 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:16.069016 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:16.069759 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:16.069760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:16.069831 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:16.069844 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 15:02:16.069923 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 15:02:16.069971 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":20,"thread_start_us":75,"threads_started":1}
I20260806 15:02:16.070057 5613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:16.066440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54284 (local address 127.11.8.65:46269)
0806 15:02:16.066691 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:16.066694 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:16.066703 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:02:16.066758 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:16.066760 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:16.066774 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:16.066881 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:16.067054 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:16.067874 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:16.069105 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:16.069674 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:16.069758 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:16.069845 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:16.069937 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 15:02:16.069972 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":80,"threads_started":1}
I20260806 15:02:16.070828 5609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:16.066089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:39997 (local address 127.0.0.1:33146)
0806 15:02:16.066461 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:16.066472 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:16.066537 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:16.067175 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:16.067176 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:16.067178 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:16.067211 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:16.067212 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:16.068460 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:16.068461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:16.069515 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:16.069517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:16.070577 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:16.070579 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:16.070645 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:16.070648 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:02:16.070708 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 15:02:16.070760 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":96,"threads_started":1}
I20260806 15:02:16.070858 5610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:16.066168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33146 (local address 127.11.8.66:39997)
0806 15:02:16.067010 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:16.067015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:16.067025 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:02:16.067066 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:16.067067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:16.067082 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:16.067163 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:16.067300 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:16.068353 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:16.069619 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:16.070489 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:16.070563 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:16.070645 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:16.070722 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 15:02:16.070771 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":798,"thread_start_us":108,"threads_started":1}
I20260806 15:02:16.070859 5177 tablet_service.cc:1467] Processing CreateTablet for tablet 1380a8a6deb44eb5b78c6dce1f472106 (DEFAULT_TABLE table=test-table [id=3a599d4ca8b24788a2e16ad4e2b6d758]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:02:16.071292 5608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:16.066061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:37581 (local address 127.0.0.1:33228)
0806 15:02:16.066387 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:16.066447 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:16.066548 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:16.066855 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:16.066859 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:16.066861 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:16.066965 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:16.066968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:16.068265 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:16.068266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:16.069427 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:16.069429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:16.070758 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:16.070759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:16.070814 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:16.070818 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:02:16.071171 (+ 353us) client_negotiation.cc:240] Negotiation successful
0806 15:02:16.071217 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":32,"thread_start_us":61,"threads_started":1}
I20260806 15:02:16.071287 5177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1380a8a6deb44eb5b78c6dce1f472106. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:16.071305 5611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:16.066220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33228 (local address 127.11.8.67:37581)
0806 15:02:16.066447 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:16.066452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:16.066514 (+ 62us) server_negotiation.cc:407] Connection header received
0806 15:02:16.066656 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:16.066658 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:16.066674 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:16.066769 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:16.067080 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:16.068165 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:16.069523 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:16.070577 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:16.070643 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:16.070726 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:16.071190 (+ 464us) server_negotiation.cc:299] Negotiation successful
0806 15:02:16.071228 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":87,"threads_started":1}
I20260806 15:02:16.073894 5616 tablet_bootstrap.cc:492] T 1380a8a6deb44eb5b78c6dce1f472106 P 48ad566658aa4bcb89c96a8a39b116df: Bootstrap starting.
I20260806 15:02:16.074641 5616 tablet_bootstrap.cc:654] T 1380a8a6deb44eb5b78c6dce1f472106 P 48ad566658aa4bcb89c96a8a39b116df: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:16.074537 5515 tablet_service.cc:1467] Processing CreateTablet for tablet 1380a8a6deb44eb5b78c6dce1f472106 (DEFAULT_TABLE table=test-table [id=3a599d4ca8b24788a2e16ad4e2b6d758]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:02:16.074893 5515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1380a8a6deb44eb5b78c6dce1f472106. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:16.074987 5616 log.cc:824] T 1380a8a6deb44eb5b78c6dce1f472106 P 48ad566658aa4bcb89c96a8a39b116df: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:16.074906 5353 tablet_service.cc:1467] Processing CreateTablet for tablet 1380a8a6deb44eb5b78c6dce1f472106 (DEFAULT_TABLE table=test-table [id=3a599d4ca8b24788a2e16ad4e2b6d758]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:02:16.075248 5353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1380a8a6deb44eb5b78c6dce1f472106. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:16.075804 5616 tablet_bootstrap.cc:492] T 1380a8a6deb44eb5b78c6dce1f472106 P 48ad566658aa4bcb89c96a8a39b116df: No bootstrap required, opened a new log
I20260806 15:02:16.075876 5616 ts_tablet_manager.cc:1397] T 1380a8a6deb44eb5b78c6dce1f472106 P 48ad566658aa4bcb89c96a8a39b116df: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:02:16.078089 5616 raft_consensus.cc:348] T 1380a8a6deb44eb5b78c6dce1f472106 P 48ad566658aa4bcb89c96a8a39b116df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0c9892845ea4715aa8e463c3c5ed025" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 39997 } } peers { permanent_uuid: "98f4ab8149b846ffb7f63d0d9c7f2582" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37581 } } peers { permanent_uuid: "48ad566658aa4bcb89c96a8a39b116df" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 46269 } }
I20260806 15:02:16.078248 5616 raft_consensus.cc:374] T 1380a8a6deb44eb5b78c6dce1f472106 P 48ad566658aa4bcb89c96a8a39b116df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:16.078293 5616 raft_consensus.cc:729] T 1380a8a6deb44eb5b78c6dce1f472106 P 48ad566658aa4bcb89c96a8a39b116df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48ad566658aa4bcb89c96a8a39b116df, State: Initialized, Role: FOLLOWER
I20260806 15:02:16.078444 5616 consensus_queue.cc:260] T 1380a8a6deb44eb5b78c6dce1f472106 P 48ad566658aa4bcb89c96a8a39b116df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0c9892845ea4715aa8e463c3c5ed025" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 39997 } } peers { permanent_uuid: "98f4ab8149b846ffb7f63d0d9c7f2582" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37581 } } peers { permanent_uuid: "48ad566658aa4bcb89c96a8a39b116df" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 46269 } }
I20260806 15:02:16.078727 5616 ts_tablet_manager.cc:1428] T 1380a8a6deb44eb5b78c6dce1f472106 P 48ad566658aa4bcb89c96a8a39b116df: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 15:02:16.078871 5262 heartbeater.cc:503] Master 127.11.8.126:34073 was elected leader, sending a full tablet report...
I20260806 15:02:16.078969 5620 tablet_bootstrap.cc:492] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025: Bootstrap starting.
I20260806 15:02:16.079527 5620 tablet_bootstrap.cc:654] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:16.079819 5620 log.cc:824] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:16.080171 5621 tablet_bootstrap.cc:492] T 1380a8a6deb44eb5b78c6dce1f472106 P 98f4ab8149b846ffb7f63d0d9c7f2582: Bootstrap starting.
I20260806 15:02:16.080603 5620 tablet_bootstrap.cc:492] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025: No bootstrap required, opened a new log
I20260806 15:02:16.080688 5620 ts_tablet_manager.cc:1397] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 15:02:16.080700 5621 tablet_bootstrap.cc:654] T 1380a8a6deb44eb5b78c6dce1f472106 P 98f4ab8149b846ffb7f63d0d9c7f2582: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:16.080919 5621 log.cc:824] T 1380a8a6deb44eb5b78c6dce1f472106 P 98f4ab8149b846ffb7f63d0d9c7f2582: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:16.081523 5621 tablet_bootstrap.cc:492] T 1380a8a6deb44eb5b78c6dce1f472106 P 98f4ab8149b846ffb7f63d0d9c7f2582: No bootstrap required, opened a new log
I20260806 15:02:16.081573 5621 ts_tablet_manager.cc:1397] T 1380a8a6deb44eb5b78c6dce1f472106 P 98f4ab8149b846ffb7f63d0d9c7f2582: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 15:02:16.082778 5620 raft_consensus.cc:348] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0c9892845ea4715aa8e463c3c5ed025" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 39997 } } peers { permanent_uuid: "98f4ab8149b846ffb7f63d0d9c7f2582" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37581 } } peers { permanent_uuid: "48ad566658aa4bcb89c96a8a39b116df" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 46269 } }
I20260806 15:02:16.082923 5620 raft_consensus.cc:374] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:16.082971 5620 raft_consensus.cc:729] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0c9892845ea4715aa8e463c3c5ed025, State: Initialized, Role: FOLLOWER
I20260806 15:02:16.083114 5620 consensus_queue.cc:260] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0c9892845ea4715aa8e463c3c5ed025" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 39997 } } peers { permanent_uuid: "98f4ab8149b846ffb7f63d0d9c7f2582" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37581 } } peers { permanent_uuid: "48ad566658aa4bcb89c96a8a39b116df" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 46269 } }
I20260806 15:02:16.083395 5620 ts_tablet_manager.cc:1428] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 15:02:16.083436 5437 heartbeater.cc:503] Master 127.11.8.126:34073 was elected leader, sending a full tablet report...
I20260806 15:02:16.083676 5621 raft_consensus.cc:348] T 1380a8a6deb44eb5b78c6dce1f472106 P 98f4ab8149b846ffb7f63d0d9c7f2582 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0c9892845ea4715aa8e463c3c5ed025" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 39997 } } peers { permanent_uuid: "98f4ab8149b846ffb7f63d0d9c7f2582" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37581 } } peers { permanent_uuid: "48ad566658aa4bcb89c96a8a39b116df" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 46269 } }
I20260806 15:02:16.083810 5621 raft_consensus.cc:374] T 1380a8a6deb44eb5b78c6dce1f472106 P 98f4ab8149b846ffb7f63d0d9c7f2582 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:16.083832 5621 raft_consensus.cc:729] T 1380a8a6deb44eb5b78c6dce1f472106 P 98f4ab8149b846ffb7f63d0d9c7f2582 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98f4ab8149b846ffb7f63d0d9c7f2582, State: Initialized, Role: FOLLOWER
I20260806 15:02:16.083904 5621 consensus_queue.cc:260] T 1380a8a6deb44eb5b78c6dce1f472106 P 98f4ab8149b846ffb7f63d0d9c7f2582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0c9892845ea4715aa8e463c3c5ed025" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 39997 } } peers { permanent_uuid: "98f4ab8149b846ffb7f63d0d9c7f2582" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37581 } } peers { permanent_uuid: "48ad566658aa4bcb89c96a8a39b116df" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 46269 } }
I20260806 15:02:16.084136 5621 ts_tablet_manager.cc:1428] T 1380a8a6deb44eb5b78c6dce1f472106 P 98f4ab8149b846ffb7f63d0d9c7f2582: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 15:02:16.084280 5597 heartbeater.cc:503] Master 127.11.8.126:34073 was elected leader, sending a full tablet report...
I20260806 15:02:16.104408 5624 raft_consensus.cc:482] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:02:16.104518 5624 raft_consensus.cc:504] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0c9892845ea4715aa8e463c3c5ed025" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 39997 } } peers { permanent_uuid: "98f4ab8149b846ffb7f63d0d9c7f2582" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37581 } } peers { permanent_uuid: "48ad566658aa4bcb89c96a8a39b116df" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 46269 } }
I20260806 15:02:16.104912 5624 leader_election.cc:288] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98f4ab8149b846ffb7f63d0d9c7f2582 (127.11.8.67:37581), 48ad566658aa4bcb89c96a8a39b116df (127.11.8.65:46269)
I20260806 15:02:16.107898 5439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:16.105105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:37581 (local address 127.11.8.66:42899)
0806 15:02:16.105166 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:16.105178 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:16.105219 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:16.105412 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:16.105413 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:16.105415 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:16.105485 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:16.105488 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:16.106197 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:16.106198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:16.107112 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:16.107115 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:16.107647 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:16.107648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:16.107719 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:16.107723 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:02:16.107788 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 15:02:16.107826 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 15:02:16.107978 5611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:16.105208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:42899 (local address 127.11.8.67:37581)
0806 15:02:16.105263 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:16.105265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:16.105274 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:02:16.105303 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:16.105304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:16.105313 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:16.105386 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:16.105559 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:16.106085 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:16.107201 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:16.107573 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:16.107630 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:16.107638 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:16.107854 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 15:02:16.107903 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 15:02:16.108172 5613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:16.105190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:45463 (local address 127.11.8.65:46269)
0806 15:02:16.105238 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:16.105240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:16.105399 (+ 159us) server_negotiation.cc:407] Connection header received
0806 15:02:16.105430 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:16.105431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:16.105444 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:16.105505 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:16.105698 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:16.106137 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:16.107269 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:16.107748 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:16.107811 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:16.107820 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:16.108053 (+ 233us) server_negotiation.cc:299] Negotiation successful
0806 15:02:16.108103 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 15:02:16.108261 5537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1380a8a6deb44eb5b78c6dce1f472106" candidate_uuid: "a0c9892845ea4715aa8e463c3c5ed025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98f4ab8149b846ffb7f63d0d9c7f2582" is_pre_election: true
I20260806 15:02:16.108393 5537 raft_consensus.cc:2454] T 1380a8a6deb44eb5b78c6dce1f472106 P 98f4ab8149b846ffb7f63d0d9c7f2582 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0c9892845ea4715aa8e463c3c5ed025 in term 0.
I20260806 15:02:16.108471 5627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:16.105110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:46269 (local address 127.11.8.66:45463)
0806 15:02:16.105316 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:16.105327 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:16.105385 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:16.105514 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:16.105515 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:16.105516 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:16.105624 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:16.105626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:16.106245 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:16.106246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:16.107174 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:16.107177 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:16.107859 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:16.107860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:16.107943 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:16.107946 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:02:16.108320 (+ 374us) client_negotiation.cc:240] Negotiation successful
0806 15:02:16.108370 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":20,"thread_start_us":119,"threads_started":1}
I20260806 15:02:16.108656 5288 leader_election.cc:302] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98f4ab8149b846ffb7f63d0d9c7f2582, a0c9892845ea4715aa8e463c3c5ed025; no voters:
I20260806 15:02:16.108790 5624 raft_consensus.cc:2781] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:02:16.108857 5624 raft_consensus.cc:482] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:02:16.108893 5624 raft_consensus.cc:3037] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:16.108853 5203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1380a8a6deb44eb5b78c6dce1f472106" candidate_uuid: "a0c9892845ea4715aa8e463c3c5ed025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48ad566658aa4bcb89c96a8a39b116df" is_pre_election: true
I20260806 15:02:16.109015 5203 raft_consensus.cc:2454] T 1380a8a6deb44eb5b78c6dce1f472106 P 48ad566658aa4bcb89c96a8a39b116df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0c9892845ea4715aa8e463c3c5ed025 in term 0.
I20260806 15:02:16.109653 5624 raft_consensus.cc:504] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0c9892845ea4715aa8e463c3c5ed025" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 39997 } } peers { permanent_uuid: "98f4ab8149b846ffb7f63d0d9c7f2582" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37581 } } peers { permanent_uuid: "48ad566658aa4bcb89c96a8a39b116df" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 46269 } }
I20260806 15:02:16.109805 5624 leader_election.cc:288] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025 [CANDIDATE]: Term 1 election: Requested vote from peers 98f4ab8149b846ffb7f63d0d9c7f2582 (127.11.8.67:37581), 48ad566658aa4bcb89c96a8a39b116df (127.11.8.65:46269)
I20260806 15:02:16.109966 5537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1380a8a6deb44eb5b78c6dce1f472106" candidate_uuid: "a0c9892845ea4715aa8e463c3c5ed025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98f4ab8149b846ffb7f63d0d9c7f2582"
I20260806 15:02:16.109999 5203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1380a8a6deb44eb5b78c6dce1f472106" candidate_uuid: "a0c9892845ea4715aa8e463c3c5ed025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48ad566658aa4bcb89c96a8a39b116df"
I20260806 15:02:16.110052 5537 raft_consensus.cc:3037] T 1380a8a6deb44eb5b78c6dce1f472106 P 98f4ab8149b846ffb7f63d0d9c7f2582 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:16.110087 5203 raft_consensus.cc:3037] T 1380a8a6deb44eb5b78c6dce1f472106 P 48ad566658aa4bcb89c96a8a39b116df [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:16.110805 5537 raft_consensus.cc:2454] T 1380a8a6deb44eb5b78c6dce1f472106 P 98f4ab8149b846ffb7f63d0d9c7f2582 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0c9892845ea4715aa8e463c3c5ed025 in term 1.
I20260806 15:02:16.110929 5203 raft_consensus.cc:2454] T 1380a8a6deb44eb5b78c6dce1f472106 P 48ad566658aa4bcb89c96a8a39b116df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0c9892845ea4715aa8e463c3c5ed025 in term 1.
I20260806 15:02:16.110980 5288 leader_election.cc:302] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98f4ab8149b846ffb7f63d0d9c7f2582, a0c9892845ea4715aa8e463c3c5ed025; no voters:
I20260806 15:02:16.111088 5624 raft_consensus.cc:2781] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:16.111227 5624 raft_consensus.cc:686] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025 [term 1 LEADER]: Becoming Leader. State: Replica: a0c9892845ea4715aa8e463c3c5ed025, State: Running, Role: LEADER
I20260806 15:02:16.111327 5624 consensus_queue.cc:237] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0c9892845ea4715aa8e463c3c5ed025" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 39997 } } peers { permanent_uuid: "98f4ab8149b846ffb7f63d0d9c7f2582" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37581 } } peers { permanent_uuid: "48ad566658aa4bcb89c96a8a39b116df" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 46269 } }
I20260806 15:02:16.112147 5012 catalog_manager.cc:5495] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0c9892845ea4715aa8e463c3c5ed025 (127.11.8.66). New cstate: current_term: 1 leader_uuid: "a0c9892845ea4715aa8e463c3c5ed025" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0c9892845ea4715aa8e463c3c5ed025" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 39997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98f4ab8149b846ffb7f63d0d9c7f2582" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48ad566658aa4bcb89c96a8a39b116df" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 46269 } health_report { overall_health: UNKNOWN } } }
I20260806 15:02:16.127972 5610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:16.125339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33160 (local address 127.11.8.66:39997)
0806 15:02:16.125388 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:16.125391 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:16.125402 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:02:16.125485 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:16.125486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:16.125513 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:16.125625 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:02:16.125793 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:16.126285 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:16.127166 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:16.127374 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:16.127457 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:16.127609 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:02:16.127704 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:02:16.127821 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:16.127865 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 15:02:16.127874 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 15:02:16.129832 5203 raft_consensus.cc:1261] T 1380a8a6deb44eb5b78c6dce1f472106 P 48ad566658aa4bcb89c96a8a39b116df [term 1 FOLLOWER]: Refusing update from remote peer a0c9892845ea4715aa8e463c3c5ed025: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:02:16.129832 5537 raft_consensus.cc:1261] T 1380a8a6deb44eb5b78c6dce1f472106 P 98f4ab8149b846ffb7f63d0d9c7f2582 [term 1 FOLLOWER]: Refusing update from remote peer a0c9892845ea4715aa8e463c3c5ed025: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:02:16.130205 5628 consensus_queue.cc:1035] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98f4ab8149b846ffb7f63d0d9c7f2582" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 37581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:02:16.130349 5624 consensus_queue.cc:1035] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48ad566658aa4bcb89c96a8a39b116df" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 46269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:02:16.135356 5013 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:02:16.135438 5013 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:02:16.136204 5013 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed02150beea94838a1216beac7966226: Sending DeleteTablet for 3 replicas of tablet 1380a8a6deb44eb5b78c6dce1f472106
I20260806 15:02:16.136580 5353 tablet_service.cc:1514] Processing DeleteTablet for tablet 1380a8a6deb44eb5b78c6dce1f472106 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:02:16 UTC) from {username='slave'} at 127.0.0.1:33146
I20260806 15:02:16.136598 5515 tablet_service.cc:1514] Processing DeleteTablet for tablet 1380a8a6deb44eb5b78c6dce1f472106 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:02:16 UTC) from {username='slave'} at 127.0.0.1:33228
I20260806 15:02:16.136644 5177 tablet_service.cc:1514] Processing DeleteTablet for tablet 1380a8a6deb44eb5b78c6dce1f472106 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:02:16 UTC) from {username='slave'} at 127.0.0.1:54284
I20260806 15:02:16.136907 5643 tablet_replica.cc:330] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025: stopping tablet replica
I20260806 15:02:16.137023 5643 raft_consensus.cc:2229] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025 [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:02:16.137123 5643 raft_consensus.cc:2258] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:02:16.137120 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 5094
I20260806 15:02:16.139819 5643 ts_tablet_manager.cc:1905] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:02:16.142554 5643 ts_tablet_manager.cc:1918] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:02:16.142643 5643 log.cc:1196] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-1/wal/wals/1380a8a6deb44eb5b78c6dce1f472106
I20260806 15:02:16.142948 5643 ts_tablet_manager.cc:1939] T 1380a8a6deb44eb5b78c6dce1f472106 P a0c9892845ea4715aa8e463c3c5ed025: Deleting consensus metadata
I20260806 15:02:16.143452 4993 catalog_manager.cc:4841] TS a0c9892845ea4715aa8e463c3c5ed025 (127.11.8.66:39997): tablet 1380a8a6deb44eb5b78c6dce1f472106 (table test-table [id=3a599d4ca8b24788a2e16ad4e2b6d758]) successfully deleted
W20260806 15:02:16.150308 4993 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.65:46269 (error 108)
I20260806 15:02:16.150439 5644 tablet_replica.cc:330] T 1380a8a6deb44eb5b78c6dce1f472106 P 98f4ab8149b846ffb7f63d0d9c7f2582: stopping tablet replica
I20260806 15:02:16.150570 5644 raft_consensus.cc:2229] T 1380a8a6deb44eb5b78c6dce1f472106 P 98f4ab8149b846ffb7f63d0d9c7f2582 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:02:16.150707 5644 raft_consensus.cc:2258] T 1380a8a6deb44eb5b78c6dce1f472106 P 98f4ab8149b846ffb7f63d0d9c7f2582 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:02:16.151109 5608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:16.150854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:46269 (local address 127.0.0.1:54294)
0806 15:02:16.151002 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:16.151068 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.65:46269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260806 15:02:16.151216 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 5266
W20260806 15:02:16.151361 4993 catalog_manager.cc:4584] TS 48ad566658aa4bcb89c96a8a39b116df (127.11.8.65:46269): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1380a8a6deb44eb5b78c6dce1f472106: Network error: Client connection negotiation failed: client connection to 127.11.8.65:46269: connect: Connection refused (error 111)
I20260806 15:02:16.158015 5644 ts_tablet_manager.cc:1905] T 1380a8a6deb44eb5b78c6dce1f472106 P 98f4ab8149b846ffb7f63d0d9c7f2582: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:02:16.160568 5644 ts_tablet_manager.cc:1918] T 1380a8a6deb44eb5b78c6dce1f472106 P 98f4ab8149b846ffb7f63d0d9c7f2582: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:02:16.160663 5644 log.cc:1196] T 1380a8a6deb44eb5b78c6dce1f472106 P 98f4ab8149b846ffb7f63d0d9c7f2582: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786028287744255-11297-0/minicluster-data/ts-2/wal/wals/1380a8a6deb44eb5b78c6dce1f472106
I20260806 15:02:16.160992 5644 ts_tablet_manager.cc:1939] T 1380a8a6deb44eb5b78c6dce1f472106 P 98f4ab8149b846ffb7f63d0d9c7f2582: Deleting consensus metadata
I20260806 15:02:16.161581 4992 catalog_manager.cc:4841] TS 98f4ab8149b846ffb7f63d0d9c7f2582 (127.11.8.67:37581): tablet 1380a8a6deb44eb5b78c6dce1f472106 (table test-table [id=3a599d4ca8b24788a2e16ad4e2b6d758]) successfully deleted
I20260806 15:02:16.167138 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 5440
I20260806 15:02:16.175709 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 4973
2026-08-06T15:02:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (712 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:02:16 dist-test-slave-68xk krb5kdc[5653](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:02:16 dist-test-slave-68xk krb5kdc[5653](info): set up 2 sockets
Aug 06 15:02:16 dist-test-slave-68xk krb5kdc[5653](info): commencing operation
krb5kdc: starting...
W20260806 15:02:17.393097 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 15:02:17 dist-test-slave-68xk krb5kdc[5653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:02:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:02:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 15:02:17.519676 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:36305
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:42259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:36305
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:17.602006 5670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:17.602270 5670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:17.602310 5670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:17.603468 5670 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:17.603538 5670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:17.603561 5670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:17.603581 5670 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:17.603600 5670 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:17.605110 5670 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:42259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:36305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:36305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:17.605394 5670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:17.605620 5670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:17.610616 5681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:17.610579 5679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:17.610617 5684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:17.610934 5670 server_base.cc:1034] running on GCE node
I20260806 15:02:17.611231 5670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:17.611562 5670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:17.612716 5670 hybrid_clock.cc:648] HybridClock initialized: now 1786028537612694 us; error 44 us; skew 500 ppm
Aug 06 15:02:17 dist-test-slave-68xk krb5kdc[5653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028537, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:17.620734 5670 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 15:02:17.621219 5670 webserver.cc:460] Webserver started at http://127.11.8.126:43549/ using document root <none> and password file <none>
I20260806 15:02:17.621475 5670 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:17.621523 5670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:17.621652 5670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:17.622785 5670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "0b4e8e4f01164348859812216ad27b19"
format_stamp: "Formatted at 2026-08-06 15:02:17 on dist-test-slave-68xk"
server_key: "ea6a711e37884c1e8f8b232d0ec7f8bf"
server_key_iv: "70510e32f0f29e6cf0b9007433602c90"
server_key_version: "encryptionkey@0"
I20260806 15:02:17.623142 5670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "0b4e8e4f01164348859812216ad27b19"
format_stamp: "Formatted at 2026-08-06 15:02:17 on dist-test-slave-68xk"
server_key: "ea6a711e37884c1e8f8b232d0ec7f8bf"
server_key_iv: "70510e32f0f29e6cf0b9007433602c90"
server_key_version: "encryptionkey@0"
I20260806 15:02:17.624502 5670 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:02:17.625313 5693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:17.625530 5670 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:02:17.625653 5670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "0b4e8e4f01164348859812216ad27b19"
format_stamp: "Formatted at 2026-08-06 15:02:17 on dist-test-slave-68xk"
server_key: "ea6a711e37884c1e8f8b232d0ec7f8bf"
server_key_iv: "70510e32f0f29e6cf0b9007433602c90"
server_key_version: "encryptionkey@0"
I20260806 15:02:17.625725 5670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:17.642341 5670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:17.644384 5670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:17.644549 5670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:17.648640 5670 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:36305
I20260806 15:02:17.648695 5763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:36305 every 8 connection(s)
I20260806 15:02:17.649072 5670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:02:17.649991 5764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:17.652339 5764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b4e8e4f01164348859812216ad27b19: Bootstrap starting.
I20260806 15:02:17.652925 5764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b4e8e4f01164348859812216ad27b19: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:17.653229 5764 log.cc:824] T 00000000000000000000000000000000 P 0b4e8e4f01164348859812216ad27b19: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:17.653924 5764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b4e8e4f01164348859812216ad27b19: No bootstrap required, opened a new log
I20260806 15:02:17.654724 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 5670
I20260806 15:02:17.654853 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:02:17.655073 11297 external_mini_cluster.cc:1428] Setting key c0405b341da26634a5a1090724edd295
I20260806 15:02:17.656183 5764 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b4e8e4f01164348859812216ad27b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b4e8e4f01164348859812216ad27b19" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 36305 } }
I20260806 15:02:17.656349 5764 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b4e8e4f01164348859812216ad27b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:17.656422 5764 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b4e8e4f01164348859812216ad27b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b4e8e4f01164348859812216ad27b19, State: Initialized, Role: FOLLOWER
I20260806 15:02:17.656591 5764 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b4e8e4f01164348859812216ad27b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b4e8e4f01164348859812216ad27b19" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 36305 } }
I20260806 15:02:17.656697 5764 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b4e8e4f01164348859812216ad27b19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:02:17.656733 5764 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b4e8e4f01164348859812216ad27b19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:02:17.656787 5764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b4e8e4f01164348859812216ad27b19 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:17.657776 5764 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b4e8e4f01164348859812216ad27b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b4e8e4f01164348859812216ad27b19" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 36305 } }
I20260806 15:02:17.657866 5764 leader_election.cc:302] T 00000000000000000000000000000000 P 0b4e8e4f01164348859812216ad27b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b4e8e4f01164348859812216ad27b19; no voters:
I20260806 15:02:17.658032 5764 leader_election.cc:288] T 00000000000000000000000000000000 P 0b4e8e4f01164348859812216ad27b19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:02:17.658135 5772 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b4e8e4f01164348859812216ad27b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:17.658319 5764 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b4e8e4f01164348859812216ad27b19 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:17.658342 5772 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b4e8e4f01164348859812216ad27b19 [term 1 LEADER]: Becoming Leader. State: Replica: 0b4e8e4f01164348859812216ad27b19, State: Running, Role: LEADER
I20260806 15:02:17.658556 5772 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b4e8e4f01164348859812216ad27b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b4e8e4f01164348859812216ad27b19" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 36305 } }
I20260806 15:02:17.659005 5773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b4e8e4f01164348859812216ad27b19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b4e8e4f01164348859812216ad27b19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b4e8e4f01164348859812216ad27b19" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 36305 } } }
I20260806 15:02:17.659090 5773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b4e8e4f01164348859812216ad27b19 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:17.659269 5775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b4e8e4f01164348859812216ad27b19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b4e8e4f01164348859812216ad27b19. Latest consensus state: current_term: 1 leader_uuid: "0b4e8e4f01164348859812216ad27b19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b4e8e4f01164348859812216ad27b19" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 36305 } } }
I20260806 15:02:17.659337 5775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b4e8e4f01164348859812216ad27b19 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:17.659795 5782 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:02:17.660354 5782 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:02:17.662385 5782 catalog_manager.cc:1347] Generated new cluster ID: 1193c2af8d894e1e9e579a3012d82ba5
I20260806 15:02:17.662475 5782 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 15:02:17 dist-test-slave-68xk krb5kdc[5653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:02:17.667825 5771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:17.655612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49710 (local address 127.11.8.126:36305)
0806 15:02:17.655789 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:17.655797 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:17.655822 (+ 25us) server_negotiation.cc:407] Connection header received
0806 15:02:17.656159 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:17.656166 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:17.656212 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:17.656379 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:17.657235 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:17.658733 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:17.660049 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:17.660665 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:17.660776 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:17.663758 (+ 2982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:17.663762 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:17.663763 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:17.663775 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:17.665858 (+ 2083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:17.666288 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:17.666292 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:17.666292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:17.666459 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:17.666701 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:17.666702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:17.666703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:17.667076 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:17.667256 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:17.667462 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 15:02:17.667623 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
I20260806 15:02:17.671643 5782 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:02:17.672101 5782 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:02:17.677064 5782 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b4e8e4f01164348859812216ad27b19: Generated new TSK 0
I20260806 15:02:17.677251 5782 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.11.8.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
Entry for principal kudu/127.11.8.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab.
I20260806 15:02:17.705127 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:0
--local_ip_for_outbound_sockets=127.11.8.65
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:36305
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:42259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:17.787886 5796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:17.788087 5796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:17.788110 5796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:17.789229 5796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:17.789278 5796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:17.789304 5796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 15:02:17.790738 5796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:42259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.65.keytab
--principal=kudu/127.11.8.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:36305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:17.790966 5796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:17.791203 5796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:17.796130 5801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:17.796200 5802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:17.796203 5806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:17.796681 5796 server_base.cc:1034] running on GCE node
I20260806 15:02:17.796941 5796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:17.797199 5796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:17.798382 5796 hybrid_clock.cc:648] HybridClock initialized: now 1786028537798362 us; error 36 us; skew 500 ppm
Aug 06 15:02:17 dist-test-slave-68xk krb5kdc[5653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028537, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:17.801266 5796 init.cc:377] Logged in from keytab as kudu/127.11.8.65@KRBTEST.COM (short username kudu)
I20260806 15:02:17.801774 5796 webserver.cc:460] Webserver started at http://127.11.8.65:42303/ using document root <none> and password file <none>
I20260806 15:02:17.802028 5796 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:17.802109 5796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:17.802274 5796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:17.803462 5796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-0/data/instance:
uuid: "5262891d590a4839a6aa68438350fce1"
format_stamp: "Formatted at 2026-08-06 15:02:17 on dist-test-slave-68xk"
server_key: "e86ac1146fa2887346891974bc576228"
server_key_iv: "50b074ad0650b0d73f80337c16ac6696"
server_key_version: "encryptionkey@0"
I20260806 15:02:17.803853 5796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance:
uuid: "5262891d590a4839a6aa68438350fce1"
format_stamp: "Formatted at 2026-08-06 15:02:17 on dist-test-slave-68xk"
server_key: "e86ac1146fa2887346891974bc576228"
server_key_iv: "50b074ad0650b0d73f80337c16ac6696"
server_key_version: "encryptionkey@0"
I20260806 15:02:17.805302 5796 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 15:02:17.806137 5818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:17.806346 5796 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:02:17.806475 5796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "5262891d590a4839a6aa68438350fce1"
format_stamp: "Formatted at 2026-08-06 15:02:17 on dist-test-slave-68xk"
server_key: "e86ac1146fa2887346891974bc576228"
server_key_iv: "50b074ad0650b0d73f80337c16ac6696"
server_key_version: "encryptionkey@0"
I20260806 15:02:17.806559 5796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:17.824101 5796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:17.826614 5796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:17.826789 5796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:17.827035 5796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:02:17.827370 5796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:02:17.827426 5796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:17.827472 5796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:02:17.827505 5796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:17.833968 5796 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:42853
I20260806 15:02:17.834038 5955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:42853 every 8 connection(s)
I20260806 15:02:17.834399 5796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
I20260806 15:02:17.840525 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 5796
I20260806 15:02:17.840647 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance
I20260806 15:02:17.840894 11297 external_mini_cluster.cc:1428] Setting key c240eb3e4588a2596ca3335e967d4802
Aug 06 15:02:17 dist-test-slave-68xk krb5kdc[5653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028537, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.65@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:02:17.845073 5771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:17.835293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:51441 (local address 127.11.8.126:36305)
0806 15:02:17.835344 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:17.835346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:17.835439 (+ 93us) server_negotiation.cc:407] Connection header received
0806 15:02:17.835970 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:17.835972 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:17.835988 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:17.836050 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:17.837124 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:17.837694 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:17.838762 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:17.838976 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:17.839061 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:17.841797 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:17.841798 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:17.841799 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:17.841808 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:17.843507 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:17.843988 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:17.843989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:17.843989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:17.844054 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:17.844339 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:17.844340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:17.844340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:17.844467 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:17.844556 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:17.844860 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 15:02:17.844961 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 15:02:17.845290 5962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:17.835136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:36305 (local address 127.11.8.65:51441)
0806 15:02:17.835313 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:17.835359 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:17.835862 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:17.836106 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:17.836109 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:17.836750 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:17.836959 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:17.836965 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:17.837810 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:17.837812 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:17.838668 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:17.838670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:17.839077 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:17.839079 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:17.839230 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:17.839782 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:02:17.839792 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:17.841688 (+ 1896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:17.843632 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:17.843641 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:17.843644 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:17.843893 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:17.844132 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:17.844132 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:17.844133 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:17.844259 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:17.844539 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:17.844547 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:17.844706 (+ 159us) client_negotiation.cc:769] Sending connection context
0806 15:02:17.844819 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 15:02:17.844997 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":100,"threads_started":1}
I20260806 15:02:17.845816 5956 heartbeater.cc:347] Connected to a master server at 127.11.8.126:36305
I20260806 15:02:17.845932 5956 heartbeater.cc:464] Registering TS with master...
I20260806 15:02:17.846163 5956 heartbeater.cc:511] Master 127.11.8.126:36305 requested a full tablet report, sending...
I20260806 15:02:17.846768 5715 ts_manager.cc:194] Registered new tserver with Master: 5262891d590a4839a6aa68438350fce1 (127.11.8.65:42853)
I20260806 15:02:17.847465 5715 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.65@KRBTEST.COM'} at 127.11.8.65:51441
WARNING: no policy specified for kudu/127.11.8.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
Entry for principal kudu/127.11.8.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab.
I20260806 15:02:17.869333 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:0
--local_ip_for_outbound_sockets=127.11.8.66
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:36305
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:42259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:17.952682 5966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:17.952894 5966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:17.952917 5966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:17.954063 5966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:17.954107 5966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:17.954133 5966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 15:02:17.955638 5966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:42259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.66.keytab
--principal=kudu/127.11.8.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:36305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:17.955888 5966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:17.956117 5966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:17.961329 5975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:17.961300 5980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:17.961391 5966 server_base.cc:1034] running on GCE node
W20260806 15:02:17.961346 5976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:17.961640 5966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:17.961879 5966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:17.963057 5966 hybrid_clock.cc:648] HybridClock initialized: now 1786028537963054 us; error 40 us; skew 500 ppm
Aug 06 15:02:17 dist-test-slave-68xk krb5kdc[5653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028537, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:17.965881 5966 init.cc:377] Logged in from keytab as kudu/127.11.8.66@KRBTEST.COM (short username kudu)
I20260806 15:02:17.966339 5966 webserver.cc:460] Webserver started at http://127.11.8.66:35387/ using document root <none> and password file <none>
I20260806 15:02:17.966599 5966 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:17.966671 5966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:17.966812 5966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:17.967828 5966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-1/data/instance:
uuid: "677e49feb65f4f09a00da73ef1915f0e"
format_stamp: "Formatted at 2026-08-06 15:02:17 on dist-test-slave-68xk"
server_key: "cb27264cb9ae2a7a8a820dbfa39afd89"
server_key_iv: "65bf0aa217ff71d70dc6fc8a4c13692f"
server_key_version: "encryptionkey@0"
I20260806 15:02:17.968216 5966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance:
uuid: "677e49feb65f4f09a00da73ef1915f0e"
format_stamp: "Formatted at 2026-08-06 15:02:17 on dist-test-slave-68xk"
server_key: "cb27264cb9ae2a7a8a820dbfa39afd89"
server_key_iv: "65bf0aa217ff71d70dc6fc8a4c13692f"
server_key_version: "encryptionkey@0"
I20260806 15:02:17.969564 5966 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:02:17.970353 5989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:17.970590 5966 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:02:17.970685 5966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "677e49feb65f4f09a00da73ef1915f0e"
format_stamp: "Formatted at 2026-08-06 15:02:17 on dist-test-slave-68xk"
server_key: "cb27264cb9ae2a7a8a820dbfa39afd89"
server_key_iv: "65bf0aa217ff71d70dc6fc8a4c13692f"
server_key_version: "encryptionkey@0"
I20260806 15:02:17.970757 5966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:17.989382 5966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:17.991530 5966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:17.991680 5966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:17.991920 5966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:02:17.992249 5966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:02:17.992285 5966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:17.992306 5966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:02:17.992319 5966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:17.998309 5966 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:44817
I20260806 15:02:17.998376 6135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:44817 every 8 connection(s)
I20260806 15:02:17.998719 5966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
I20260806 15:02:18.004551 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 5966
I20260806 15:02:18.004698 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance
I20260806 15:02:18.004906 11297 external_mini_cluster.cc:1428] Setting key e10d0c6693840050a0a8279589b0d7a3
Aug 06 15:02:17 dist-test-slave-68xk krb5kdc[5653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028537, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.66@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:02:18.009138 5771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:17.999476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:49819 (local address 127.11.8.126:36305)
0806 15:02:17.999565 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:17.999567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:17.999768 (+ 201us) server_negotiation.cc:407] Connection header received
0806 15:02:18.000321 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:18.000322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:18.000333 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:18.000395 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:18.001399 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.001921 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.003154 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.003347 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.003414 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.005798 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:18.005800 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:18.005801 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:18.005809 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:18.007566 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:18.008102 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:18.008103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:18.008103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:18.008167 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:18.008465 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:18.008466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:18.008466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:18.008576 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:18.008633 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:18.008954 (+ 321us) server_negotiation.cc:299] Negotiation successful
0806 15:02:18.009072 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 15:02:18.009431 6139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:17.999441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:36305 (local address 127.11.8.66:49819)
0806 15:02:17.999637 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:17.999677 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:18.000199 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:18.000455 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:18.000458 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:18.001099 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:18.001301 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:18.001306 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.002035 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.002036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:18.003060 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:18.003062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.003430 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.003431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:18.003539 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.004012 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:02:18.004021 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:18.005720 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:18.007665 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:18.007673 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:18.007676 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:18.008002 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:18.008245 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:18.008246 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:18.008246 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:18.008395 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:18.008639 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:18.008646 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:18.008809 (+ 163us) client_negotiation.cc:769] Sending connection context
0806 15:02:18.008936 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 15:02:18.009116 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":97,"threads_started":1}
I20260806 15:02:18.009927 6137 heartbeater.cc:347] Connected to a master server at 127.11.8.126:36305
I20260806 15:02:18.010042 6137 heartbeater.cc:464] Registering TS with master...
I20260806 15:02:18.010218 6137 heartbeater.cc:511] Master 127.11.8.126:36305 requested a full tablet report, sending...
I20260806 15:02:18.010660 5715 ts_manager.cc:194] Registered new tserver with Master: 677e49feb65f4f09a00da73ef1915f0e (127.11.8.66:44817)
I20260806 15:02:18.011191 5715 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.66@KRBTEST.COM'} at 127.11.8.66:49819
WARNING: no policy specified for kudu/127.11.8.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
Entry for principal kudu/127.11.8.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab.
I20260806 15:02:18.033351 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:0
--local_ip_for_outbound_sockets=127.11.8.67
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:36305
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:42259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:18.119084 6142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:18.119279 6142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:18.119302 6142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:18.120467 6142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:18.120515 6142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:18.120541 6142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 15:02:18.121976 6142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:42259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.67.keytab
--principal=kudu/127.11.8.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:36305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:18.122239 6142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:18.122493 6142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:18.127252 6148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:18.127297 6149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:18.127264 6151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:18.127674 6142 server_base.cc:1034] running on GCE node
I20260806 15:02:18.127866 6142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:18.128149 6142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:18.129309 6142 hybrid_clock.cc:648] HybridClock initialized: now 1786028538129300 us; error 38 us; skew 500 ppm
Aug 06 15:02:18 dist-test-slave-68xk krb5kdc[5653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028538, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:18.131902 6142 init.cc:377] Logged in from keytab as kudu/127.11.8.67@KRBTEST.COM (short username kudu)
I20260806 15:02:18.132362 6142 webserver.cc:460] Webserver started at http://127.11.8.67:38033/ using document root <none> and password file <none>
I20260806 15:02:18.132547 6142 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:18.132591 6142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:18.132679 6142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:18.133651 6142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-2/data/instance:
uuid: "5c3e4a4b15ce4cb69e5ef79ed27ddce9"
format_stamp: "Formatted at 2026-08-06 15:02:18 on dist-test-slave-68xk"
server_key: "9c92aaef561e2406e499db7d3143d687"
server_key_iv: "b791d416cd8b01f0bf5e6dc05757810d"
server_key_version: "encryptionkey@0"
I20260806 15:02:18.133953 6142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance:
uuid: "5c3e4a4b15ce4cb69e5ef79ed27ddce9"
format_stamp: "Formatted at 2026-08-06 15:02:18 on dist-test-slave-68xk"
server_key: "9c92aaef561e2406e499db7d3143d687"
server_key_iv: "b791d416cd8b01f0bf5e6dc05757810d"
server_key_version: "encryptionkey@0"
I20260806 15:02:18.135353 6142 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:02:18.136031 6160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:18.136242 6142 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:02:18.136332 6142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "5c3e4a4b15ce4cb69e5ef79ed27ddce9"
format_stamp: "Formatted at 2026-08-06 15:02:18 on dist-test-slave-68xk"
server_key: "9c92aaef561e2406e499db7d3143d687"
server_key_iv: "b791d416cd8b01f0bf5e6dc05757810d"
server_key_version: "encryptionkey@0"
I20260806 15:02:18.136384 6142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:18.151732 6142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:18.153690 6142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:18.153831 6142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:18.154080 6142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:02:18.154389 6142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:02:18.154467 6142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:18.154505 6142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:02:18.154531 6142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:18.162139 6142 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:38145
I20260806 15:02:18.162298 6315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:38145 every 8 connection(s)
I20260806 15:02:18.162575 6142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
Aug 06 15:02:18 dist-test-slave-68xk krb5kdc[5653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028538, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.67@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:02:18.169514 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 6142
I20260806 15:02:18.169656 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance
I20260806 15:02:18.169878 11297 external_mini_cluster.cc:1428] Setting key b6b880c57c340e2cceb3f1571b69fcad
I20260806 15:02:18.171926 5771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.163268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:37539 (local address 127.11.8.126:36305)
0806 15:02:18.163325 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:18.163327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:18.163543 (+ 216us) server_negotiation.cc:407] Connection header received
0806 15:02:18.164022 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:18.164023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:18.164035 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:18.164091 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:18.164992 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.165546 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.166524 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.166732 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.166790 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.168953 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:18.168955 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:18.168955 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:18.168964 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:18.170865 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:18.171217 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:18.171218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:18.171218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:18.171273 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:18.171453 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:18.171454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:18.171454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:18.171558 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:18.171624 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:18.171801 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 15:02:18.171860 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 15:02:18.172086 6320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.163282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:36305 (local address 127.11.8.67:37539)
0806 15:02:18.163440 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:18.163475 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:18.163930 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:18.164143 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:18.164146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:18.164729 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:18.164913 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:18.164918 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.165626 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.165626 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:18.166402 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:18.166405 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.166805 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.166806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:18.166908 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.167354 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:02:18.167362 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:18.168855 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:18.170926 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:18.170932 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:18.170933 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:18.171151 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:18.171324 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:18.171324 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:18.171324 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:18.171405 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:18.171607 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:18.171610 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:18.171712 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 15:02:18.171784 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 15:02:18.171897 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
I20260806 15:02:18.172551 6316 heartbeater.cc:347] Connected to a master server at 127.11.8.126:36305
I20260806 15:02:18.172667 6316 heartbeater.cc:464] Registering TS with master...
I20260806 15:02:18.172855 6316 heartbeater.cc:511] Master 127.11.8.126:36305 requested a full tablet report, sending...
I20260806 15:02:18.173238 5715 ts_manager.cc:194] Registered new tserver with Master: 5c3e4a4b15ce4cb69e5ef79ed27ddce9 (127.11.8.67:38145)
I20260806 15:02:18.173655 5715 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.11.8.67@KRBTEST.COM'} at 127.11.8.67:37539
I20260806 15:02:18.181452 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:02:18.188799 5771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.182830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49726 (local address 127.11.8.126:36305)
0806 15:02:18.182922 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:18.182924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:18.183033 (+ 109us) server_negotiation.cc:407] Connection header received
0806 15:02:18.183106 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:18.183107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:18.183117 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:18.183166 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:18.183737 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.184206 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.184936 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.185185 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.185267 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.186379 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:18.186380 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:18.186380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:18.186388 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:18.187630 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:18.187964 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:18.187965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:18.187966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:18.188032 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:18.188226 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:18.188226 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:18.188227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:18.188369 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:18.188471 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:18.188627 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 15:02:18.188703 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 15:02:18.190241 5715 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:02:18.191838 5715 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:02:18.198585 6334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.194488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39574 (local address 127.11.8.66:44817)
0806 15:02:18.194646 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:18.194650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:18.195086 (+ 436us) server_negotiation.cc:407] Connection header received
0806 15:02:18.195137 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:18.195139 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:18.195156 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:18.195232 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:18.195344 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.196012 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.197515 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.198195 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.198260 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.198308 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:18.198456 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 15:02:18.198504 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
I20260806 15:02:18.198647 6332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.194154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:42853 (local address 127.0.0.1:40414)
0806 15:02:18.194344 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:18.194381 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:18.194458 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:18.195128 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:18.195128 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:18.195131 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:18.195166 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:18.195167 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.196443 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.196444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:18.197321 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:18.197324 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.198207 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.198208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:18.198303 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.198319 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 15:02:18.198401 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 15:02:18.198485 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":4,"thread_start_us":67,"threads_started":1}
I20260806 15:02:18.198647 6330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.194199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:44817 (local address 127.0.0.1:39574)
0806 15:02:18.194961 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:18.194971 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:18.195023 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:18.195230 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:18.195231 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:18.195233 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:18.195286 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:18.195287 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.196116 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.196117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:18.197418 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:18.197421 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.198273 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.198274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:18.198322 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.198326 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:02:18.198401 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 15:02:18.198487 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"mutex_wait_us":27,"thread_start_us":104,"threads_started":1}
I20260806 15:02:18.198899 6333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.194442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40414 (local address 127.11.8.65:42853)
0806 15:02:18.194777 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:18.194781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:18.194789 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:02:18.194820 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:18.194822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:18.194850 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:18.195240 (+ 390us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:18.195314 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.196346 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.197408 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.198117 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.198197 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.198260 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:18.198774 (+ 514us) server_negotiation.cc:299] Negotiation successful
0806 15:02:18.198827 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":81,"threads_started":1}
I20260806 15:02:18.199441 6048 tablet_service.cc:1467] Processing CreateTablet for tablet b370411814124f8283c0ea1f3c5dd6c2 (DEFAULT_TABLE table=test-table [id=86eb707c4cc54676bf4bd765cef95e1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:02:18.199812 6048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b370411814124f8283c0ea1f3c5dd6c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:18.199754 5875 tablet_service.cc:1467] Processing CreateTablet for tablet b370411814124f8283c0ea1f3c5dd6c2 (DEFAULT_TABLE table=test-table [id=86eb707c4cc54676bf4bd765cef95e1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:02:18.199998 6335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.194623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39358 (local address 127.11.8.67:38145)
0806 15:02:18.194763 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:18.194766 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:18.194774 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:02:18.194805 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:18.194807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:18.194819 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:18.194921 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:18.195116 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.196235 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.198346 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.199636 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.199711 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.199768 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:18.199876 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 15:02:18.199917 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
I20260806 15:02:18.200132 5875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b370411814124f8283c0ea1f3c5dd6c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:18.200253 6331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.194153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:38145 (local address 127.0.0.1:39358)
0806 15:02:18.194321 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:18.194360 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:18.194450 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:18.194944 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:18.194948 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:18.194950 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:18.195030 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:18.195032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.196340 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.196341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:18.197421 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:18.197422 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.199715 (+ 2293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.199716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:18.199784 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.199789 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 15:02:18.199984 (+ 195us) client_negotiation.cc:240] Negotiation successful
0806 15:02:18.200159 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":61,"threads_started":1}
I20260806 15:02:18.202454 6336 tablet_bootstrap.cc:492] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e: Bootstrap starting.
I20260806 15:02:18.202664 6337 tablet_bootstrap.cc:492] T b370411814124f8283c0ea1f3c5dd6c2 P 5262891d590a4839a6aa68438350fce1: Bootstrap starting.
I20260806 15:02:18.203315 6336 tablet_bootstrap.cc:654] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:18.203346 6337 tablet_bootstrap.cc:654] T b370411814124f8283c0ea1f3c5dd6c2 P 5262891d590a4839a6aa68438350fce1: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:18.203642 6230 tablet_service.cc:1467] Processing CreateTablet for tablet b370411814124f8283c0ea1f3c5dd6c2 (DEFAULT_TABLE table=test-table [id=86eb707c4cc54676bf4bd765cef95e1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:02:18.203810 6336 log.cc:824] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:18.203893 6337 log.cc:824] T b370411814124f8283c0ea1f3c5dd6c2 P 5262891d590a4839a6aa68438350fce1: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:18.204003 6230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b370411814124f8283c0ea1f3c5dd6c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:18.204582 6337 tablet_bootstrap.cc:492] T b370411814124f8283c0ea1f3c5dd6c2 P 5262891d590a4839a6aa68438350fce1: No bootstrap required, opened a new log
I20260806 15:02:18.204632 6336 tablet_bootstrap.cc:492] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e: No bootstrap required, opened a new log
I20260806 15:02:18.204664 6337 ts_tablet_manager.cc:1397] T b370411814124f8283c0ea1f3c5dd6c2 P 5262891d590a4839a6aa68438350fce1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:02:18.204702 6336 ts_tablet_manager.cc:1397] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:02:18.206208 6337 raft_consensus.cc:348] T b370411814124f8283c0ea1f3c5dd6c2 P 5262891d590a4839a6aa68438350fce1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3e4a4b15ce4cb69e5ef79ed27ddce9" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38145 } } peers { permanent_uuid: "5262891d590a4839a6aa68438350fce1" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 42853 } } peers { permanent_uuid: "677e49feb65f4f09a00da73ef1915f0e" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44817 } }
I20260806 15:02:18.206368 6337 raft_consensus.cc:374] T b370411814124f8283c0ea1f3c5dd6c2 P 5262891d590a4839a6aa68438350fce1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:18.206414 6337 raft_consensus.cc:729] T b370411814124f8283c0ea1f3c5dd6c2 P 5262891d590a4839a6aa68438350fce1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5262891d590a4839a6aa68438350fce1, State: Initialized, Role: FOLLOWER
I20260806 15:02:18.206576 6337 consensus_queue.cc:260] T b370411814124f8283c0ea1f3c5dd6c2 P 5262891d590a4839a6aa68438350fce1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3e4a4b15ce4cb69e5ef79ed27ddce9" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38145 } } peers { permanent_uuid: "5262891d590a4839a6aa68438350fce1" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 42853 } } peers { permanent_uuid: "677e49feb65f4f09a00da73ef1915f0e" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44817 } }
I20260806 15:02:18.206863 6337 ts_tablet_manager.cc:1428] T b370411814124f8283c0ea1f3c5dd6c2 P 5262891d590a4839a6aa68438350fce1: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 15:02:18.206777 6336 raft_consensus.cc:348] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3e4a4b15ce4cb69e5ef79ed27ddce9" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38145 } } peers { permanent_uuid: "5262891d590a4839a6aa68438350fce1" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 42853 } } peers { permanent_uuid: "677e49feb65f4f09a00da73ef1915f0e" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44817 } }
I20260806 15:02:18.206879 5956 heartbeater.cc:503] Master 127.11.8.126:36305 was elected leader, sending a full tablet report...
I20260806 15:02:18.206944 6336 raft_consensus.cc:374] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:18.206982 6336 raft_consensus.cc:729] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 677e49feb65f4f09a00da73ef1915f0e, State: Initialized, Role: FOLLOWER
I20260806 15:02:18.207119 6336 consensus_queue.cc:260] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3e4a4b15ce4cb69e5ef79ed27ddce9" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38145 } } peers { permanent_uuid: "5262891d590a4839a6aa68438350fce1" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 42853 } } peers { permanent_uuid: "677e49feb65f4f09a00da73ef1915f0e" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44817 } }
I20260806 15:02:18.207407 6336 ts_tablet_manager.cc:1428] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 15:02:18.207551 6137 heartbeater.cc:503] Master 127.11.8.126:36305 was elected leader, sending a full tablet report...
I20260806 15:02:18.207931 6344 tablet_bootstrap.cc:492] T b370411814124f8283c0ea1f3c5dd6c2 P 5c3e4a4b15ce4cb69e5ef79ed27ddce9: Bootstrap starting.
I20260806 15:02:18.208652 6344 tablet_bootstrap.cc:654] T b370411814124f8283c0ea1f3c5dd6c2 P 5c3e4a4b15ce4cb69e5ef79ed27ddce9: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:18.209244 6344 log.cc:824] T b370411814124f8283c0ea1f3c5dd6c2 P 5c3e4a4b15ce4cb69e5ef79ed27ddce9: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:18.209976 6344 tablet_bootstrap.cc:492] T b370411814124f8283c0ea1f3c5dd6c2 P 5c3e4a4b15ce4cb69e5ef79ed27ddce9: No bootstrap required, opened a new log
I20260806 15:02:18.210029 6344 ts_tablet_manager.cc:1397] T b370411814124f8283c0ea1f3c5dd6c2 P 5c3e4a4b15ce4cb69e5ef79ed27ddce9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:02:18.211454 6344 raft_consensus.cc:348] T b370411814124f8283c0ea1f3c5dd6c2 P 5c3e4a4b15ce4cb69e5ef79ed27ddce9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3e4a4b15ce4cb69e5ef79ed27ddce9" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38145 } } peers { permanent_uuid: "5262891d590a4839a6aa68438350fce1" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 42853 } } peers { permanent_uuid: "677e49feb65f4f09a00da73ef1915f0e" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44817 } }
I20260806 15:02:18.211584 6344 raft_consensus.cc:374] T b370411814124f8283c0ea1f3c5dd6c2 P 5c3e4a4b15ce4cb69e5ef79ed27ddce9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:18.211606 6344 raft_consensus.cc:729] T b370411814124f8283c0ea1f3c5dd6c2 P 5c3e4a4b15ce4cb69e5ef79ed27ddce9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3e4a4b15ce4cb69e5ef79ed27ddce9, State: Initialized, Role: FOLLOWER
I20260806 15:02:18.211726 6344 consensus_queue.cc:260] T b370411814124f8283c0ea1f3c5dd6c2 P 5c3e4a4b15ce4cb69e5ef79ed27ddce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3e4a4b15ce4cb69e5ef79ed27ddce9" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38145 } } peers { permanent_uuid: "5262891d590a4839a6aa68438350fce1" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 42853 } } peers { permanent_uuid: "677e49feb65f4f09a00da73ef1915f0e" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44817 } }
I20260806 15:02:18.211975 6344 ts_tablet_manager.cc:1428] T b370411814124f8283c0ea1f3c5dd6c2 P 5c3e4a4b15ce4cb69e5ef79ed27ddce9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 15:02:18.212028 6316 heartbeater.cc:503] Master 127.11.8.126:36305 was elected leader, sending a full tablet report...
I20260806 15:02:18.225881 6343 raft_consensus.cc:482] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:02:18.225991 6343 raft_consensus.cc:504] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3e4a4b15ce4cb69e5ef79ed27ddce9" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38145 } } peers { permanent_uuid: "5262891d590a4839a6aa68438350fce1" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 42853 } } peers { permanent_uuid: "677e49feb65f4f09a00da73ef1915f0e" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44817 } }
I20260806 15:02:18.226367 6343 leader_election.cc:288] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c3e4a4b15ce4cb69e5ef79ed27ddce9 (127.11.8.67:38145), 5262891d590a4839a6aa68438350fce1 (127.11.8.65:42853)
I20260806 15:02:18.229583 6139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.226548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:38145 (local address 127.11.8.66:60653)
0806 15:02:18.226634 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:18.226648 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:18.226704 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:18.226830 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:18.226831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:18.226833 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:18.226912 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:18.226914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.227699 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.227699 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:18.228465 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:18.228467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.229306 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.229307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:18.229388 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.229393 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 15:02:18.229461 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 15:02:18.229505 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 15:02:18.229584 6335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.226592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:60653 (local address 127.11.8.67:38145)
0806 15:02:18.226651 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:18.226654 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:18.226686 (+ 32us) server_negotiation.cc:407] Connection header received
0806 15:02:18.226759 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:18.226760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:18.226772 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:18.226824 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:18.227003 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.227621 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.228548 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.229245 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.229302 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.229311 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:18.229468 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 15:02:18.229495 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 15:02:18.229858 6349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.226555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:42853 (local address 127.11.8.66:35201)
0806 15:02:18.226873 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:18.226885 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:18.226969 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:18.227098 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:18.227098 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:18.227100 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:18.227168 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:18.227169 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.227783 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.227784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:18.228892 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:18.228895 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.229566 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.229567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:18.229653 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.229657 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:02:18.229729 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 15:02:18.229768 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":129,"thread_start_us":136,"threads_started":1}
I20260806 15:02:18.229869 6333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.226656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:35201 (local address 127.11.8.65:42853)
0806 15:02:18.226705 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:18.226707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:18.226961 (+ 254us) server_negotiation.cc:407] Connection header received
0806 15:02:18.227026 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:18.227027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:18.227037 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:18.227087 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:18.227229 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.227701 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.228986 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.229484 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.229547 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.229575 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:18.229746 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 15:02:18.229795 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 15:02:18.230046 6249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b370411814124f8283c0ea1f3c5dd6c2" candidate_uuid: "677e49feb65f4f09a00da73ef1915f0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3e4a4b15ce4cb69e5ef79ed27ddce9" is_pre_election: true
I20260806 15:02:18.230186 6249 raft_consensus.cc:2454] T b370411814124f8283c0ea1f3c5dd6c2 P 5c3e4a4b15ce4cb69e5ef79ed27ddce9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 677e49feb65f4f09a00da73ef1915f0e in term 0.
I20260806 15:02:18.230448 5991 leader_election.cc:302] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3e4a4b15ce4cb69e5ef79ed27ddce9, 677e49feb65f4f09a00da73ef1915f0e; no voters:
I20260806 15:02:18.230588 6343 raft_consensus.cc:2781] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:02:18.230561 5900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b370411814124f8283c0ea1f3c5dd6c2" candidate_uuid: "677e49feb65f4f09a00da73ef1915f0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5262891d590a4839a6aa68438350fce1" is_pre_election: true
I20260806 15:02:18.230645 6343 raft_consensus.cc:482] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:02:18.230680 6343 raft_consensus.cc:3037] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:18.230711 5900 raft_consensus.cc:2454] T b370411814124f8283c0ea1f3c5dd6c2 P 5262891d590a4839a6aa68438350fce1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 677e49feb65f4f09a00da73ef1915f0e in term 0.
I20260806 15:02:18.231413 6343 raft_consensus.cc:504] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3e4a4b15ce4cb69e5ef79ed27ddce9" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38145 } } peers { permanent_uuid: "5262891d590a4839a6aa68438350fce1" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 42853 } } peers { permanent_uuid: "677e49feb65f4f09a00da73ef1915f0e" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44817 } }
I20260806 15:02:18.231552 6343 leader_election.cc:288] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e [CANDIDATE]: Term 1 election: Requested vote from peers 5c3e4a4b15ce4cb69e5ef79ed27ddce9 (127.11.8.67:38145), 5262891d590a4839a6aa68438350fce1 (127.11.8.65:42853)
I20260806 15:02:18.231714 6249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b370411814124f8283c0ea1f3c5dd6c2" candidate_uuid: "677e49feb65f4f09a00da73ef1915f0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c3e4a4b15ce4cb69e5ef79ed27ddce9"
I20260806 15:02:18.231760 5900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b370411814124f8283c0ea1f3c5dd6c2" candidate_uuid: "677e49feb65f4f09a00da73ef1915f0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5262891d590a4839a6aa68438350fce1"
I20260806 15:02:18.231791 6249 raft_consensus.cc:3037] T b370411814124f8283c0ea1f3c5dd6c2 P 5c3e4a4b15ce4cb69e5ef79ed27ddce9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:18.231848 5900 raft_consensus.cc:3037] T b370411814124f8283c0ea1f3c5dd6c2 P 5262891d590a4839a6aa68438350fce1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:18.232522 6249 raft_consensus.cc:2454] T b370411814124f8283c0ea1f3c5dd6c2 P 5c3e4a4b15ce4cb69e5ef79ed27ddce9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 677e49feb65f4f09a00da73ef1915f0e in term 1.
I20260806 15:02:18.232699 5900 raft_consensus.cc:2454] T b370411814124f8283c0ea1f3c5dd6c2 P 5262891d590a4839a6aa68438350fce1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 677e49feb65f4f09a00da73ef1915f0e in term 1.
I20260806 15:02:18.232717 5991 leader_election.cc:302] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c3e4a4b15ce4cb69e5ef79ed27ddce9, 677e49feb65f4f09a00da73ef1915f0e; no voters:
I20260806 15:02:18.232890 6343 raft_consensus.cc:2781] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:18.233065 6343 raft_consensus.cc:686] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e [term 1 LEADER]: Becoming Leader. State: Replica: 677e49feb65f4f09a00da73ef1915f0e, State: Running, Role: LEADER
I20260806 15:02:18.233176 6343 consensus_queue.cc:237] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3e4a4b15ce4cb69e5ef79ed27ddce9" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38145 } } peers { permanent_uuid: "5262891d590a4839a6aa68438350fce1" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 42853 } } peers { permanent_uuid: "677e49feb65f4f09a00da73ef1915f0e" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44817 } }
I20260806 15:02:18.234017 5714 catalog_manager.cc:5495] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e reported cstate change: term changed from 0 to 1, leader changed from <none> to 677e49feb65f4f09a00da73ef1915f0e (127.11.8.66). New cstate: current_term: 1 leader_uuid: "677e49feb65f4f09a00da73ef1915f0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c3e4a4b15ce4cb69e5ef79ed27ddce9" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5262891d590a4839a6aa68438350fce1" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 42853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "677e49feb65f4f09a00da73ef1915f0e" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 44817 } health_report { overall_health: HEALTHY } } }
I20260806 15:02:18.247345 6334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.244892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39580 (local address 127.11.8.66:44817)
0806 15:02:18.244964 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:18.244967 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:18.244996 (+ 29us) server_negotiation.cc:407] Connection header received
0806 15:02:18.245035 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:18.245035 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:18.245054 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:18.245114 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:02:18.245231 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.245673 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.246541 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.246728 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.246783 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.246996 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:02:18.247093 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:02:18.247204 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:18.247218 (+ 14us) server_negotiation.cc:299] Negotiation successful
0806 15:02:18.247234 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 15:02:18.249244 6138 tablet.cc:2369] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:02:18.249342 6249 raft_consensus.cc:1261] T b370411814124f8283c0ea1f3c5dd6c2 P 5c3e4a4b15ce4cb69e5ef79ed27ddce9 [term 1 FOLLOWER]: Refusing update from remote peer 677e49feb65f4f09a00da73ef1915f0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:02:18.249373 5900 raft_consensus.cc:1261] T b370411814124f8283c0ea1f3c5dd6c2 P 5262891d590a4839a6aa68438350fce1 [term 1 FOLLOWER]: Refusing update from remote peer 677e49feb65f4f09a00da73ef1915f0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:02:18.249786 6350 consensus_queue.cc:1035] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c3e4a4b15ce4cb69e5ef79ed27ddce9" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 38145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:02:18.249939 6343 consensus_queue.cc:1035] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5262891d590a4839a6aa68438350fce1" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 42853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:02:18.252660 6355 mvcc.cc:205] Tried to move back new op lower bound from 7315572892665987072 to 7315572892603969536. Current Snapshot: MvccSnapshot[applied={T|T < 7315572892665987072}]
I20260806 15:02:18.253461 6356 mvcc.cc:205] Tried to move back new op lower bound from 7315572892665987072 to 7315572892603969536. Current Snapshot: MvccSnapshot[applied={T|T < 7315572892665987072 or (T in {7315572892665987072})}]
I20260806 15:02:18.255154 5715 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:02:18.255234 5715 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:02:18.255976 5715 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b4e8e4f01164348859812216ad27b19: Sending DeleteTablet for 3 replicas of tablet b370411814124f8283c0ea1f3c5dd6c2
I20260806 15:02:18.256289 6230 tablet_service.cc:1514] Processing DeleteTablet for tablet b370411814124f8283c0ea1f3c5dd6c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:02:18 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:39358
I20260806 15:02:18.256307 5875 tablet_service.cc:1514] Processing DeleteTablet for tablet b370411814124f8283c0ea1f3c5dd6c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:02:18 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:40414
I20260806 15:02:18.256377 6048 tablet_service.cc:1514] Processing DeleteTablet for tablet b370411814124f8283c0ea1f3c5dd6c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:02:18 UTC) from {username='kudu', principal='kudu/127.11.8.126@KRBTEST.COM'} at 127.0.0.1:39574
I20260806 15:02:18.256524 6368 tablet_replica.cc:330] T b370411814124f8283c0ea1f3c5dd6c2 P 5262891d590a4839a6aa68438350fce1: stopping tablet replica
I20260806 15:02:18.256633 6368 raft_consensus.cc:2229] T b370411814124f8283c0ea1f3c5dd6c2 P 5262891d590a4839a6aa68438350fce1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:02:18.256739 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 5796
I20260806 15:02:18.270565 6369 tablet_replica.cc:330] T b370411814124f8283c0ea1f3c5dd6c2 P 5c3e4a4b15ce4cb69e5ef79ed27ddce9: stopping tablet replica
I20260806 15:02:18.270714 6369 raft_consensus.cc:2229] T b370411814124f8283c0ea1f3c5dd6c2 P 5c3e4a4b15ce4cb69e5ef79ed27ddce9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:02:18.270824 6369 raft_consensus.cc:2258] T b370411814124f8283c0ea1f3c5dd6c2 P 5c3e4a4b15ce4cb69e5ef79ed27ddce9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:02:18.271411 6369 ts_tablet_manager.cc:1905] T b370411814124f8283c0ea1f3c5dd6c2 P 5c3e4a4b15ce4cb69e5ef79ed27ddce9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:02:18.276621 6369 ts_tablet_manager.cc:1918] T b370411814124f8283c0ea1f3c5dd6c2 P 5c3e4a4b15ce4cb69e5ef79ed27ddce9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:02:18.276798 6369 log.cc:1196] T b370411814124f8283c0ea1f3c5dd6c2 P 5c3e4a4b15ce4cb69e5ef79ed27ddce9: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-2/wal/wals/b370411814124f8283c0ea1f3c5dd6c2
I20260806 15:02:18.276888 6371 tablet_replica.cc:330] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e: stopping tablet replica
I20260806 15:02:18.277014 6371 raft_consensus.cc:2229] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:02:18.277137 6369 ts_tablet_manager.cc:1939] T b370411814124f8283c0ea1f3c5dd6c2 P 5c3e4a4b15ce4cb69e5ef79ed27ddce9: Deleting consensus metadata
I20260806 15:02:18.277163 6371 raft_consensus.cc:2258] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:02:18.277644 6371 ts_tablet_manager.cc:1905] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:02:18.277629 5696 catalog_manager.cc:4841] TS 5c3e4a4b15ce4cb69e5ef79ed27ddce9 (127.11.8.67:38145): tablet b370411814124f8283c0ea1f3c5dd6c2 (table test-table [id=86eb707c4cc54676bf4bd765cef95e1b]) successfully deleted
I20260806 15:02:18.279012 6371 ts_tablet_manager.cc:1918] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:02:18.279106 6371 log.cc:1196] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786028287744255-11297-0/minicluster-data/ts-1/wal/wals/b370411814124f8283c0ea1f3c5dd6c2
I20260806 15:02:18.279361 6371 ts_tablet_manager.cc:1939] T b370411814124f8283c0ea1f3c5dd6c2 P 677e49feb65f4f09a00da73ef1915f0e: Deleting consensus metadata
I20260806 15:02:18.279763 5698 catalog_manager.cc:4841] TS 677e49feb65f4f09a00da73ef1915f0e (127.11.8.66:44817): tablet b370411814124f8283c0ea1f3c5dd6c2 (table test-table [id=86eb707c4cc54676bf4bd765cef95e1b]) successfully deleted
W20260806 15:02:18.283145 5694 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.65:42853 (error 108)
I20260806 15:02:18.283324 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 5966
I20260806 15:02:18.285243 6331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.283611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:42853 (local address 127.0.0.1:40430)
0806 15:02:18.285167 (+ 1556us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:18.285213 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.65:42853: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1531}
W20260806 15:02:18.287185 5694 catalog_manager.cc:4584] TS 5262891d590a4839a6aa68438350fce1 (127.11.8.65:42853): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b370411814124f8283c0ea1f3c5dd6c2: Network error: Client connection negotiation failed: client connection to 127.11.8.65:42853: connect: Connection refused (error 111)
I20260806 15:02:18.294312 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 6142
I20260806 15:02:18.303980 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 5670
2026-08-06T15:02:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2129 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T15:02:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:02:18Z Disabled control of system clock
I20260806 15:02:18.336526 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:36307
--webserver_interface=127.11.8.126
--webserver_port=0
--builtin_ntp_servers=127.11.8.84:40527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:36307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 15:02:18.417851 6376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:18.418069 6376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:18.418099 6376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:18.419214 6376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:18.419250 6376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:18.419262 6376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:18.419275 6376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:18.419287 6376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:18.420542 6376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:40527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:36307
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:36307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:18.420737 6376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:18.420969 6376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:18.425716 6388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:18.425698 6384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:18.425675 6385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:18.425937 6376 server_base.cc:1034] running on GCE node
I20260806 15:02:18.426174 6376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:18.426405 6376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:18.427605 6376 hybrid_clock.cc:648] HybridClock initialized: now 1786028538427591 us; error 30 us; skew 500 ppm
I20260806 15:02:18.428821 6376 webserver.cc:460] Webserver started at http://127.11.8.126:46133/ using document root <none> and password file <none>
I20260806 15:02:18.429051 6376 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:18.429148 6376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:18.429296 6376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:18.430353 6376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "a584547a3b9f4ccd99ca73e87f6d3b15"
format_stamp: "Formatted at 2026-08-06 15:02:18 on dist-test-slave-68xk"
server_key: "0801e8323fdd0046daf5eb337db984ef"
server_key_iv: "cedde54c064691d5699ca070fd7fa575"
server_key_version: "encryptionkey@0"
I20260806 15:02:18.430778 6376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "a584547a3b9f4ccd99ca73e87f6d3b15"
format_stamp: "Formatted at 2026-08-06 15:02:18 on dist-test-slave-68xk"
server_key: "0801e8323fdd0046daf5eb337db984ef"
server_key_iv: "cedde54c064691d5699ca070fd7fa575"
server_key_version: "encryptionkey@0"
I20260806 15:02:18.432097 6376 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:02:18.432845 6394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:18.433028 6376 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:02:18.433130 6376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "a584547a3b9f4ccd99ca73e87f6d3b15"
format_stamp: "Formatted at 2026-08-06 15:02:18 on dist-test-slave-68xk"
server_key: "0801e8323fdd0046daf5eb337db984ef"
server_key_iv: "cedde54c064691d5699ca070fd7fa575"
server_key_version: "encryptionkey@0"
I20260806 15:02:18.433213 6376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:18.440816 6376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:18.441124 6376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:18.441270 6376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:18.445416 6376 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:36307
I20260806 15:02:18.445449 6462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:36307 every 8 connection(s)
I20260806 15:02:18.445803 6376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:02:18.446861 6463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:18.449108 6463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a584547a3b9f4ccd99ca73e87f6d3b15: Bootstrap starting.
I20260806 15:02:18.449685 6463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a584547a3b9f4ccd99ca73e87f6d3b15: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:18.449983 6463 log.cc:824] T 00000000000000000000000000000000 P a584547a3b9f4ccd99ca73e87f6d3b15: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:18.450788 6463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a584547a3b9f4ccd99ca73e87f6d3b15: No bootstrap required, opened a new log
I20260806 15:02:18.451031 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 6376
I20260806 15:02:18.451153 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:02:18.451388 11297 external_mini_cluster.cc:1428] Setting key 222bc21815f72a6cf0dfc1195793aec5
I20260806 15:02:18.452953 6463 raft_consensus.cc:348] T 00000000000000000000000000000000 P a584547a3b9f4ccd99ca73e87f6d3b15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a584547a3b9f4ccd99ca73e87f6d3b15" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 36307 } }
I20260806 15:02:18.453176 6463 raft_consensus.cc:374] T 00000000000000000000000000000000 P a584547a3b9f4ccd99ca73e87f6d3b15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:18.453231 6463 raft_consensus.cc:729] T 00000000000000000000000000000000 P a584547a3b9f4ccd99ca73e87f6d3b15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a584547a3b9f4ccd99ca73e87f6d3b15, State: Initialized, Role: FOLLOWER
I20260806 15:02:18.453359 6463 consensus_queue.cc:260] T 00000000000000000000000000000000 P a584547a3b9f4ccd99ca73e87f6d3b15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a584547a3b9f4ccd99ca73e87f6d3b15" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 36307 } }
I20260806 15:02:18.453459 6463 raft_consensus.cc:388] T 00000000000000000000000000000000 P a584547a3b9f4ccd99ca73e87f6d3b15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:02:18.453511 6463 raft_consensus.cc:482] T 00000000000000000000000000000000 P a584547a3b9f4ccd99ca73e87f6d3b15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:02:18.453560 6463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a584547a3b9f4ccd99ca73e87f6d3b15 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:18.454082 6463 raft_consensus.cc:504] T 00000000000000000000000000000000 P a584547a3b9f4ccd99ca73e87f6d3b15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a584547a3b9f4ccd99ca73e87f6d3b15" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 36307 } }
I20260806 15:02:18.454203 6463 leader_election.cc:302] T 00000000000000000000000000000000 P a584547a3b9f4ccd99ca73e87f6d3b15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a584547a3b9f4ccd99ca73e87f6d3b15; no voters:
I20260806 15:02:18.454398 6463 leader_election.cc:288] T 00000000000000000000000000000000 P a584547a3b9f4ccd99ca73e87f6d3b15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:02:18.454488 6469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a584547a3b9f4ccd99ca73e87f6d3b15 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:18.454674 6469 raft_consensus.cc:686] T 00000000000000000000000000000000 P a584547a3b9f4ccd99ca73e87f6d3b15 [term 1 LEADER]: Becoming Leader. State: Replica: a584547a3b9f4ccd99ca73e87f6d3b15, State: Running, Role: LEADER
I20260806 15:02:18.454717 6463 sys_catalog.cc:564] T 00000000000000000000000000000000 P a584547a3b9f4ccd99ca73e87f6d3b15 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:18.454766 6469 consensus_queue.cc:237] T 00000000000000000000000000000000 P a584547a3b9f4ccd99ca73e87f6d3b15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a584547a3b9f4ccd99ca73e87f6d3b15" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 36307 } }
I20260806 15:02:18.455108 6469 sys_catalog.cc:455] T 00000000000000000000000000000000 P a584547a3b9f4ccd99ca73e87f6d3b15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a584547a3b9f4ccd99ca73e87f6d3b15. Latest consensus state: current_term: 1 leader_uuid: "a584547a3b9f4ccd99ca73e87f6d3b15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a584547a3b9f4ccd99ca73e87f6d3b15" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 36307 } } }
I20260806 15:02:18.455195 6469 sys_catalog.cc:458] T 00000000000000000000000000000000 P a584547a3b9f4ccd99ca73e87f6d3b15 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:18.455889 6470 sys_catalog.cc:455] T 00000000000000000000000000000000 P a584547a3b9f4ccd99ca73e87f6d3b15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a584547a3b9f4ccd99ca73e87f6d3b15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a584547a3b9f4ccd99ca73e87f6d3b15" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 36307 } } }
I20260806 15:02:18.455969 6470 sys_catalog.cc:458] T 00000000000000000000000000000000 P a584547a3b9f4ccd99ca73e87f6d3b15 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:18.456146 6483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:02:18.456887 6483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:02:18.459353 6483 catalog_manager.cc:1347] Generated new cluster ID: 6340ed983f6e4f1c82fd404fc7516f8e
I20260806 15:02:18.459410 6483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:02:18.459786 6467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.451934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43982 (local address 127.11.8.126:36307)
0806 15:02:18.452129 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:18.452136 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:18.452161 (+ 25us) server_negotiation.cc:407] Connection header received
0806 15:02:18.452933 (+ 772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:18.452940 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:18.452998 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:18.453134 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:18.453392 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.454940 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.456850 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.457883 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.458004 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.458970 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:18.458974 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:18.458974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:18.459197 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:18.459248 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:18.459265 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:18.459335 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:18.459512 (+ 177us) server_negotiation.cc:299] Negotiation successful
0806 15:02:18.459614 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
I20260806 15:02:18.491699 6483 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:02:18.492465 6483 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:02:18.499682 6483 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a584547a3b9f4ccd99ca73e87f6d3b15: Generated new TSK 0
I20260806 15:02:18.499883 6483 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:02:18.507046 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.65:0
--local_ip_for_outbound_sockets=127.11.8.65
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:36307
--builtin_ntp_servers=127.11.8.84:40527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 15:02:18.600064 6493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:18.600281 6493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:18.600303 6493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:18.601451 6493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:18.601492 6493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:18.601517 6493 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.65
I20260806 15:02:18.603035 6493 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:40527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.11.8.65
--webserver_port=0
--tserver_master_addrs=127.11.8.126:36307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.65
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:18.603267 6493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:18.603591 6493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:18.608785 6500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:18.608762 6502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:18.608786 6493 server_base.cc:1034] running on GCE node
W20260806 15:02:18.608770 6507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:18.609261 6493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:18.609546 6493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:18.610700 6493 hybrid_clock.cc:648] HybridClock initialized: now 1786028538610688 us; error 28 us; skew 500 ppm
I20260806 15:02:18.612004 6493 webserver.cc:460] Webserver started at http://127.11.8.65:41525/ using document root <none> and password file <none>
I20260806 15:02:18.612272 6493 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:18.612342 6493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:18.612493 6493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:18.613587 6493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-0/data/instance:
uuid: "d2974b69f8524717b7612e13f7149d55"
format_stamp: "Formatted at 2026-08-06 15:02:18 on dist-test-slave-68xk"
server_key: "c9905a4e85296e28cc39bedee413fdf2"
server_key_iv: "627a15e380e8867a83fdc284a7b7adcc"
server_key_version: "encryptionkey@0"
I20260806 15:02:18.613940 6493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance:
uuid: "d2974b69f8524717b7612e13f7149d55"
format_stamp: "Formatted at 2026-08-06 15:02:18 on dist-test-slave-68xk"
server_key: "c9905a4e85296e28cc39bedee413fdf2"
server_key_iv: "627a15e380e8867a83fdc284a7b7adcc"
server_key_version: "encryptionkey@0"
I20260806 15:02:18.615337 6493 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:02:18.616041 6515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:18.616266 6493 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:02:18.616370 6493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-0/wal
uuid: "d2974b69f8524717b7612e13f7149d55"
format_stamp: "Formatted at 2026-08-06 15:02:18 on dist-test-slave-68xk"
server_key: "c9905a4e85296e28cc39bedee413fdf2"
server_key_iv: "627a15e380e8867a83fdc284a7b7adcc"
server_key_version: "encryptionkey@0"
I20260806 15:02:18.616439 6493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:18.635661 6493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:18.636057 6493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:18.636232 6493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:18.636492 6493 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:02:18.636839 6493 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:02:18.636893 6493 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:18.636937 6493 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:02:18.636976 6493 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:18.642968 6493 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.65:32831
I20260806 15:02:18.643020 6659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.65:32831 every 8 connection(s)
I20260806 15:02:18.643406 6493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-0/data/info.pb
I20260806 15:02:18.649019 6467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.644149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.65:58495 (local address 127.11.8.126:36307)
0806 15:02:18.644201 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:18.644204 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:18.644481 (+ 277us) server_negotiation.cc:407] Connection header received
0806 15:02:18.645013 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:18.645015 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:18.645067 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:18.645144 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:18.645883 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.646498 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.647500 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.647708 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.647772 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.648693 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:18.648694 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:18.648695 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:18.648704 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:18.648721 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:18.648731 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:18.648797 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:18.648905 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 15:02:18.648937 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 15:02:18.649242 6664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.644128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:36307 (local address 127.11.8.65:58495)
0806 15:02:18.644336 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:18.644381 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:18.644885 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:18.645228 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:18.645238 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:18.645247 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:18.645775 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:18.645780 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.646598 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.646599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:18.647410 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:18.647413 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.647792 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.647792 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:18.647903 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.648513 (+ 610us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:02:18.648514 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:18.648565 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:02:18.648590 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:18.648797 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:18.648805 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:18.648809 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 15:02:18.648892 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 15:02:18.649002 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":100,"threads_started":1}
I20260806 15:02:18.649807 6661 heartbeater.cc:347] Connected to a master server at 127.11.8.126:36307
I20260806 15:02:18.649893 6661 heartbeater.cc:464] Registering TS with master...
I20260806 15:02:18.650092 6661 heartbeater.cc:511] Master 127.11.8.126:36307 requested a full tablet report, sending...
I20260806 15:02:18.650632 6417 ts_manager.cc:194] Registered new tserver with Master: d2974b69f8524717b7612e13f7149d55 (127.11.8.65:32831)
I20260806 15:02:18.651273 6417 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.8.65:58495
I20260806 15:02:18.653498 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 6493
I20260806 15:02:18.653587 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-0/wal/instance
I20260806 15:02:18.653753 11297 external_mini_cluster.cc:1428] Setting key e3ba7064af034402e61394f4ce39d7d8
I20260806 15:02:18.654748 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.66:0
--local_ip_for_outbound_sockets=127.11.8.66
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:36307
--builtin_ntp_servers=127.11.8.84:40527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 15:02:18.738401 6665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:18.738623 6665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:18.738646 6665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:18.739722 6665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:18.739758 6665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:18.739784 6665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.66
I20260806 15:02:18.741083 6665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:40527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.11.8.66
--webserver_port=0
--tserver_master_addrs=127.11.8.126:36307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.66
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:18.741276 6665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:18.741523 6665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:18.746709 6677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:18.746729 6665 server_base.cc:1034] running on GCE node
W20260806 15:02:18.746733 6674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:18.746712 6673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:18.747212 6665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:18.747481 6665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:18.748672 6665 hybrid_clock.cc:648] HybridClock initialized: now 1786028538748661 us; error 37 us; skew 500 ppm
I20260806 15:02:18.750186 6665 webserver.cc:460] Webserver started at http://127.11.8.66:42999/ using document root <none> and password file <none>
I20260806 15:02:18.750524 6665 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:18.750581 6665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:18.750723 6665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:18.751801 6665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-1/data/instance:
uuid: "816466dc509349318085c9fbb52035ab"
format_stamp: "Formatted at 2026-08-06 15:02:18 on dist-test-slave-68xk"
server_key: "2b6c14256426b03fd393b6e14ffbe079"
server_key_iv: "9c64cf486816dc4ae2893e4c33079198"
server_key_version: "encryptionkey@0"
I20260806 15:02:18.752182 6665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance:
uuid: "816466dc509349318085c9fbb52035ab"
format_stamp: "Formatted at 2026-08-06 15:02:18 on dist-test-slave-68xk"
server_key: "2b6c14256426b03fd393b6e14ffbe079"
server_key_iv: "9c64cf486816dc4ae2893e4c33079198"
server_key_version: "encryptionkey@0"
I20260806 15:02:18.753475 6665 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:02:18.754325 6684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:18.754510 6665 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:02:18.754606 6665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-1/wal
uuid: "816466dc509349318085c9fbb52035ab"
format_stamp: "Formatted at 2026-08-06 15:02:18 on dist-test-slave-68xk"
server_key: "2b6c14256426b03fd393b6e14ffbe079"
server_key_iv: "9c64cf486816dc4ae2893e4c33079198"
server_key_version: "encryptionkey@0"
I20260806 15:02:18.754678 6665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:18.779611 6665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:18.779918 6665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:18.780076 6665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:18.780342 6665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:02:18.780686 6665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:02:18.780721 6665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:18.780761 6665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:02:18.780790 6665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:18.786375 6665 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.66:41941
I20260806 15:02:18.786489 6830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.66:41941 every 8 connection(s)
I20260806 15:02:18.786796 6665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-1/data/info.pb
I20260806 15:02:18.790473 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 6665
I20260806 15:02:18.790601 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-1/wal/instance
I20260806 15:02:18.790817 11297 external_mini_cluster.cc:1428] Setting key 01463e0f4e0c9a15f9b99ccb65d1ca53
I20260806 15:02:18.791918 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.11.8.67:0
--local_ip_for_outbound_sockets=127.11.8.67
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:36307
--builtin_ntp_servers=127.11.8.84:40527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 15:02:18.792162 6467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.787489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:60647 (local address 127.11.8.126:36307)
0806 15:02:18.787535 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:18.787537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:18.787798 (+ 261us) server_negotiation.cc:407] Connection header received
0806 15:02:18.788283 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:18.788285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:18.788295 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:18.788347 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:18.788978 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.789472 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.790518 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.790810 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.790889 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.791856 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:18.791858 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:18.791858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:18.791862 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:18.791877 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:18.791881 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:18.791921 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:18.792027 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 15:02:18.792064 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:02:18.792449 6834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.787481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:36307 (local address 127.11.8.66:60647)
0806 15:02:18.787652 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:18.787705 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:18.788199 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:18.788399 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:18.788402 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:18.788411 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:18.788904 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:18.788907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.789552 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.789552 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:18.790367 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:18.790370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.790897 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.790898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:18.791079 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.791690 (+ 611us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:02:18.791691 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:18.791731 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:02:18.791761 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:18.791937 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:18.791942 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:18.791945 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 15:02:18.792021 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 15:02:18.792140 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":89,"threads_started":1}
I20260806 15:02:18.792954 6832 heartbeater.cc:347] Connected to a master server at 127.11.8.126:36307
I20260806 15:02:18.793053 6832 heartbeater.cc:464] Registering TS with master...
I20260806 15:02:18.793242 6832 heartbeater.cc:511] Master 127.11.8.126:36307 requested a full tablet report, sending...
I20260806 15:02:18.793625 6417 ts_manager.cc:194] Registered new tserver with Master: 816466dc509349318085c9fbb52035ab (127.11.8.66:41941)
I20260806 15:02:18.793991 6417 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.8.66:60647
W20260806 15:02:18.875771 6835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:18.876036 6835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:18.876071 6835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:18.877250 6835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:18.877314 6835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:18.877357 6835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.11.8.67
I20260806 15:02:18.878768 6835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:40527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.11.8.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.11.8.67
--webserver_port=0
--tserver_master_addrs=127.11.8.126:36307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.11.8.67
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:18.879033 6835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:18.879350 6835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:18.884855 6842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:18.885078 6845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:18.884876 6843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:18.884874 6835 server_base.cc:1034] running on GCE node
I20260806 15:02:18.885442 6835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:18.885746 6835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:18.886898 6835 hybrid_clock.cc:648] HybridClock initialized: now 1786028538886876 us; error 41 us; skew 500 ppm
I20260806 15:02:18.888244 6835 webserver.cc:460] Webserver started at http://127.11.8.67:40455/ using document root <none> and password file <none>
I20260806 15:02:18.888520 6835 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:18.888566 6835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:18.888723 6835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:18.889832 6835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-2/data/instance:
uuid: "7cccbf0ac00049ec9a9fb11e6bf8e7a7"
format_stamp: "Formatted at 2026-08-06 15:02:18 on dist-test-slave-68xk"
server_key: "13354345d799f77d742c44964dab240d"
server_key_iv: "668c7a1555ba6f5310de87fccff9dacf"
server_key_version: "encryptionkey@0"
I20260806 15:02:18.890215 6835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance:
uuid: "7cccbf0ac00049ec9a9fb11e6bf8e7a7"
format_stamp: "Formatted at 2026-08-06 15:02:18 on dist-test-slave-68xk"
server_key: "13354345d799f77d742c44964dab240d"
server_key_iv: "668c7a1555ba6f5310de87fccff9dacf"
server_key_version: "encryptionkey@0"
I20260806 15:02:18.891662 6835 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:02:18.892422 6854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:18.892602 6835 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:02:18.892697 6835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-2/wal
uuid: "7cccbf0ac00049ec9a9fb11e6bf8e7a7"
format_stamp: "Formatted at 2026-08-06 15:02:18 on dist-test-slave-68xk"
server_key: "13354345d799f77d742c44964dab240d"
server_key_iv: "668c7a1555ba6f5310de87fccff9dacf"
server_key_version: "encryptionkey@0"
I20260806 15:02:18.892781 6835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:18.906242 6835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:18.906658 6835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:18.906812 6835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:18.907040 6835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 15:02:18.907384 6835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 15:02:18.907437 6835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:18.907480 6835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 15:02:18.907508 6835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:18.913501 6835 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.67:41473
I20260806 15:02:18.913554 6994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.67:41473 every 8 connection(s)
I20260806 15:02:18.913875 6835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-2/data/info.pb
I20260806 15:02:18.918367 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 6835
I20260806 15:02:18.918612 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-2/wal/instance
I20260806 15:02:18.918928 11297 external_mini_cluster.cc:1428] Setting key 391f696ffdb3dd575e066ebc67810e27
I20260806 15:02:18.920149 6467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.914626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.67:58867 (local address 127.11.8.126:36307)
0806 15:02:18.914701 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:18.914704 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:18.914949 (+ 245us) server_negotiation.cc:407] Connection header received
0806 15:02:18.915440 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:18.915441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:18.915451 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:18.915508 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:18.916234 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.916740 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.917769 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.917961 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.918017 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.919599 (+ 1582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:18.919601 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:18.919602 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:18.919619 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:18.919648 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:18.919654 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:18.919816 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:18.919948 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 15:02:18.919994 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 15:02:18.920607 6998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.914617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:36307 (local address 127.11.8.67:58867)
0806 15:02:18.914823 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:18.914858 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:18.915350 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:18.915584 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:18.915588 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:18.915596 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:18.916135 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:18.916141 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.916858 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.916859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:18.917681 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:18.917683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.918038 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.918039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:18.918223 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.919257 (+ 1034us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 15:02:18.919258 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:18.919306 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 15:02:18.919429 (+ 123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:18.919816 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:18.919832 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:18.919837 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 15:02:18.919953 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 15:02:18.920169 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":100,"threads_started":1}
I20260806 15:02:18.921341 6996 heartbeater.cc:347] Connected to a master server at 127.11.8.126:36307
I20260806 15:02:18.921523 6996 heartbeater.cc:464] Registering TS with master...
I20260806 15:02:18.921801 6996 heartbeater.cc:511] Master 127.11.8.126:36307 requested a full tablet report, sending...
I20260806 15:02:18.922375 6417 ts_manager.cc:194] Registered new tserver with Master: 7cccbf0ac00049ec9a9fb11e6bf8e7a7 (127.11.8.67:41473)
I20260806 15:02:18.922957 6417 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.11.8.67:58867
I20260806 15:02:18.931975 11297 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 15:02:18.939852 6467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.934405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43992 (local address 127.11.8.126:36307)
0806 15:02:18.934521 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:18.934523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:18.934762 (+ 239us) server_negotiation.cc:407] Connection header received
0806 15:02:18.934925 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:18.934926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:18.934969 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:18.935083 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:18.935397 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.936322 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.937614 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.937932 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.938061 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.939325 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:18.939328 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:18.939329 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:18.939335 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:18.939359 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:18.939384 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:18.939512 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:18.939659 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 15:02:18.939705 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 15:02:18.942528 6417 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43992:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 15:02:18.945267 6417 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 15:02:18.956593 7007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.949113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:32831 (local address 127.0.0.1:48798)
0806 15:02:18.950147 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:18.950169 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:18.950338 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:18.950944 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:18.950945 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:18.950947 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:18.951013 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:18.951015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.952682 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.952683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:18.954338 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:18.954341 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.956140 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.956141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:18.956262 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.956289 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 15:02:18.956393 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 15:02:18.956487 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":992,"thread_start_us":140,"threads_started":1}
I20260806 15:02:18.957007 7006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.949243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48798 (local address 127.11.8.65:32831)
0806 15:02:18.949592 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:18.949613 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:18.950278 (+ 665us) server_negotiation.cc:407] Connection header received
0806 15:02:18.950546 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:18.950549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:18.950603 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:18.950737 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:18.951216 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.952462 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.954521 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.955833 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.955945 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.956083 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:18.956797 (+ 714us) server_negotiation.cc:299] Negotiation successful
0806 15:02:18.956877 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":185,"threads_started":1}
I20260806 15:02:18.957089 7004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.949047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:41473 (local address 127.0.0.1:41276)
0806 15:02:18.949627 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:18.949711 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:18.949849 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:18.950340 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:18.950344 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:18.950347 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:18.950610 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:18.950613 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.953452 (+ 2839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.953453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:18.955169 (+ 1716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:18.955171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.956791 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.956792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:18.956883 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.956893 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 15:02:18.956939 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 15:02:18.956997 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":113,"threads_started":1}
I20260806 15:02:18.957834 7008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.949453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41276 (local address 127.11.8.67:41473)
0806 15:02:18.949800 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:18.949804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:18.949842 (+ 38us) server_negotiation.cc:407] Connection header received
0806 15:02:18.950013 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:18.950019 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:18.950070 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:18.950842 (+ 772us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:18.950973 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.953257 (+ 2284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.955345 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.956653 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.957300 (+ 647us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.957477 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:18.957507 (+ 30us) server_negotiation.cc:299] Negotiation successful
0806 15:02:18.957678 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":205,"threads_started":1}
I20260806 15:02:18.958310 6577 tablet_service.cc:1467] Processing CreateTablet for tablet 717e8a75c20e46ac9c606af8da69b487 (DEFAULT_TABLE table=test-table [id=84f57bbec02048f2bb9aada582d54da6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:02:18.958639 7005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.949458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.66:41941 (local address 127.0.0.1:46612)
0806 15:02:18.950242 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:18.950263 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:18.950402 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:18.950816 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:18.950817 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:18.950829 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:18.950945 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:18.950947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.954657 (+ 3710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.954663 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:18.956201 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:18.956203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.958087 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.958088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:18.958177 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.958183 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 15:02:18.958340 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 15:02:18.958462 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"thread_start_us":99,"threads_started":1}
I20260806 15:02:18.958685 7009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:18.949820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46612 (local address 127.11.8.66:41941)
0806 15:02:18.950129 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:18.950145 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:18.950345 (+ 200us) server_negotiation.cc:407] Connection header received
0806 15:02:18.950604 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:18.950608 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:18.950664 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:18.950840 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:18.951063 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.953279 (+ 2216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.956324 (+ 3045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:18.957758 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:18.957896 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:18.958035 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:18.958444 (+ 409us) server_negotiation.cc:299] Negotiation successful
0806 15:02:18.958515 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":146,"threads_started":1}
I20260806 15:02:18.959121 6577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 717e8a75c20e46ac9c606af8da69b487. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:18.959383 6916 tablet_service.cc:1467] Processing CreateTablet for tablet 717e8a75c20e46ac9c606af8da69b487 (DEFAULT_TABLE table=test-table [id=84f57bbec02048f2bb9aada582d54da6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:02:18.959946 6916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 717e8a75c20e46ac9c606af8da69b487. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:18.960152 6745 tablet_service.cc:1467] Processing CreateTablet for tablet 717e8a75c20e46ac9c606af8da69b487 (DEFAULT_TABLE table=test-table [id=84f57bbec02048f2bb9aada582d54da6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 15:02:18.960676 6745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 717e8a75c20e46ac9c606af8da69b487. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:18.963907 7010 tablet_bootstrap.cc:492] T 717e8a75c20e46ac9c606af8da69b487 P 7cccbf0ac00049ec9a9fb11e6bf8e7a7: Bootstrap starting.
I20260806 15:02:18.964828 7011 tablet_bootstrap.cc:492] T 717e8a75c20e46ac9c606af8da69b487 P d2974b69f8524717b7612e13f7149d55: Bootstrap starting.
I20260806 15:02:18.965178 7010 tablet_bootstrap.cc:654] T 717e8a75c20e46ac9c606af8da69b487 P 7cccbf0ac00049ec9a9fb11e6bf8e7a7: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:18.965651 7010 log.cc:824] T 717e8a75c20e46ac9c606af8da69b487 P 7cccbf0ac00049ec9a9fb11e6bf8e7a7: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:18.965880 7011 tablet_bootstrap.cc:654] T 717e8a75c20e46ac9c606af8da69b487 P d2974b69f8524717b7612e13f7149d55: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:18.966073 7012 tablet_bootstrap.cc:492] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab: Bootstrap starting.
I20260806 15:02:18.966486 7011 log.cc:824] T 717e8a75c20e46ac9c606af8da69b487 P d2974b69f8524717b7612e13f7149d55: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:18.966931 7010 tablet_bootstrap.cc:492] T 717e8a75c20e46ac9c606af8da69b487 P 7cccbf0ac00049ec9a9fb11e6bf8e7a7: No bootstrap required, opened a new log
I20260806 15:02:18.967067 7010 ts_tablet_manager.cc:1397] T 717e8a75c20e46ac9c606af8da69b487 P 7cccbf0ac00049ec9a9fb11e6bf8e7a7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 15:02:18.967245 7012 tablet_bootstrap.cc:654] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:18.967713 7012 log.cc:824] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:18.967831 7011 tablet_bootstrap.cc:492] T 717e8a75c20e46ac9c606af8da69b487 P d2974b69f8524717b7612e13f7149d55: No bootstrap required, opened a new log
I20260806 15:02:18.967947 7011 ts_tablet_manager.cc:1397] T 717e8a75c20e46ac9c606af8da69b487 P d2974b69f8524717b7612e13f7149d55: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 15:02:18.968693 7012 tablet_bootstrap.cc:492] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab: No bootstrap required, opened a new log
I20260806 15:02:18.968806 7012 ts_tablet_manager.cc:1397] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 15:02:18.971618 7010 raft_consensus.cc:348] T 717e8a75c20e46ac9c606af8da69b487 P 7cccbf0ac00049ec9a9fb11e6bf8e7a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cccbf0ac00049ec9a9fb11e6bf8e7a7" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41473 } } peers { permanent_uuid: "d2974b69f8524717b7612e13f7149d55" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 32831 } } peers { permanent_uuid: "816466dc509349318085c9fbb52035ab" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41941 } }
I20260806 15:02:18.971936 7010 raft_consensus.cc:374] T 717e8a75c20e46ac9c606af8da69b487 P 7cccbf0ac00049ec9a9fb11e6bf8e7a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:18.971809 7012 raft_consensus.cc:348] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cccbf0ac00049ec9a9fb11e6bf8e7a7" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41473 } } peers { permanent_uuid: "d2974b69f8524717b7612e13f7149d55" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 32831 } } peers { permanent_uuid: "816466dc509349318085c9fbb52035ab" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41941 } }
I20260806 15:02:18.972028 7012 raft_consensus.cc:374] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:18.972009 7010 raft_consensus.cc:729] T 717e8a75c20e46ac9c606af8da69b487 P 7cccbf0ac00049ec9a9fb11e6bf8e7a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cccbf0ac00049ec9a9fb11e6bf8e7a7, State: Initialized, Role: FOLLOWER
I20260806 15:02:18.972090 7012 raft_consensus.cc:729] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 816466dc509349318085c9fbb52035ab, State: Initialized, Role: FOLLOWER
I20260806 15:02:18.972335 7012 consensus_queue.cc:260] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cccbf0ac00049ec9a9fb11e6bf8e7a7" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41473 } } peers { permanent_uuid: "d2974b69f8524717b7612e13f7149d55" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 32831 } } peers { permanent_uuid: "816466dc509349318085c9fbb52035ab" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41941 } }
I20260806 15:02:18.972352 7010 consensus_queue.cc:260] T 717e8a75c20e46ac9c606af8da69b487 P 7cccbf0ac00049ec9a9fb11e6bf8e7a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cccbf0ac00049ec9a9fb11e6bf8e7a7" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41473 } } peers { permanent_uuid: "d2974b69f8524717b7612e13f7149d55" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 32831 } } peers { permanent_uuid: "816466dc509349318085c9fbb52035ab" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41941 } }
I20260806 15:02:18.972224 7011 raft_consensus.cc:348] T 717e8a75c20e46ac9c606af8da69b487 P d2974b69f8524717b7612e13f7149d55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cccbf0ac00049ec9a9fb11e6bf8e7a7" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41473 } } peers { permanent_uuid: "d2974b69f8524717b7612e13f7149d55" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 32831 } } peers { permanent_uuid: "816466dc509349318085c9fbb52035ab" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41941 } }
I20260806 15:02:18.972474 7011 raft_consensus.cc:374] T 717e8a75c20e46ac9c606af8da69b487 P d2974b69f8524717b7612e13f7149d55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:18.972570 7011 raft_consensus.cc:729] T 717e8a75c20e46ac9c606af8da69b487 P d2974b69f8524717b7612e13f7149d55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2974b69f8524717b7612e13f7149d55, State: Initialized, Role: FOLLOWER
I20260806 15:02:18.972780 7010 ts_tablet_manager.cc:1428] T 717e8a75c20e46ac9c606af8da69b487 P 7cccbf0ac00049ec9a9fb11e6bf8e7a7: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260806 15:02:18.972811 7012 ts_tablet_manager.cc:1428] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260806 15:02:18.972803 7011 consensus_queue.cc:260] T 717e8a75c20e46ac9c606af8da69b487 P d2974b69f8524717b7612e13f7149d55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cccbf0ac00049ec9a9fb11e6bf8e7a7" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41473 } } peers { permanent_uuid: "d2974b69f8524717b7612e13f7149d55" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 32831 } } peers { permanent_uuid: "816466dc509349318085c9fbb52035ab" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41941 } }
I20260806 15:02:18.973037 6832 heartbeater.cc:503] Master 127.11.8.126:36307 was elected leader, sending a full tablet report...
I20260806 15:02:18.973011 6996 heartbeater.cc:503] Master 127.11.8.126:36307 was elected leader, sending a full tablet report...
I20260806 15:02:18.973228 7011 ts_tablet_manager.cc:1428] T 717e8a75c20e46ac9c606af8da69b487 P d2974b69f8524717b7612e13f7149d55: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 15:02:18.973449 6661 heartbeater.cc:503] Master 127.11.8.126:36307 was elected leader, sending a full tablet report...
I20260806 15:02:19.001024 7018 raft_consensus.cc:482] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 15:02:19.001294 7018 raft_consensus.cc:504] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cccbf0ac00049ec9a9fb11e6bf8e7a7" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41473 } } peers { permanent_uuid: "d2974b69f8524717b7612e13f7149d55" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 32831 } } peers { permanent_uuid: "816466dc509349318085c9fbb52035ab" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41941 } }
I20260806 15:02:19.002041 7018 leader_election.cc:288] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cccbf0ac00049ec9a9fb11e6bf8e7a7 (127.11.8.67:41473), d2974b69f8524717b7612e13f7149d55 (127.11.8.65:32831)
I20260806 15:02:19.007835 6834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:19.002286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.67:41473 (local address 127.11.8.66:42477)
0806 15:02:19.002397 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:19.002437 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:19.002530 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:19.002881 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:19.002882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:19.002884 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:19.003032 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:19.003034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:19.003948 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:19.003949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:19.005827 (+ 1878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:19.005830 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:19.007310 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:19.007311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:19.007455 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:19.007469 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 15:02:19.007566 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 15:02:19.007657 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260806 15:02:19.008354 7008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:19.002476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:42477 (local address 127.11.8.67:41473)
0806 15:02:19.002702 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:19.002704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:19.002712 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:02:19.002767 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:19.002768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:19.002786 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:19.002879 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:19.003145 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:19.003812 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:19.006043 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:19.007076 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:19.007228 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:19.007243 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:19.008146 (+ 903us) server_negotiation.cc:299] Negotiation successful
0806 15:02:19.008248 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187}
I20260806 15:02:19.009068 6940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "717e8a75c20e46ac9c606af8da69b487" candidate_uuid: "816466dc509349318085c9fbb52035ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cccbf0ac00049ec9a9fb11e6bf8e7a7" is_pre_election: true
I20260806 15:02:19.009447 7006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:19.002505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.11.8.66:57697 (local address 127.11.8.65:32831)
0806 15:02:19.002610 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:19.002623 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:19.002877 (+ 254us) server_negotiation.cc:407] Connection header received
0806 15:02:19.003069 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:19.003071 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:19.003093 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:19.003222 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 15:02:19.003489 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:19.004320 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:19.007364 (+ 3044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:19.008502 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:19.008672 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:19.008696 (+ 24us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:19.009178 (+ 482us) server_negotiation.cc:299] Negotiation successful
0806 15:02:19.009260 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 15:02:19.009526 6940 raft_consensus.cc:2454] T 717e8a75c20e46ac9c606af8da69b487 P 7cccbf0ac00049ec9a9fb11e6bf8e7a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 816466dc509349318085c9fbb52035ab in term 0.
I20260806 15:02:19.010056 7020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:19.002287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:32831 (local address 127.11.8.66:57697)
0806 15:02:19.002772 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:19.002791 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:19.002928 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:19.003228 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:19.003229 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:19.003231 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 15:02:19.003353 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:19.003364 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:19.004531 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:19.004533 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:19.007106 (+ 2573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:19.007109 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:19.008788 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:19.008789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:19.008967 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:19.008973 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 15:02:19.009704 (+ 731us) client_negotiation.cc:240] Negotiation successful
0806 15:02:19.009812 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":117,"thread_start_us":241,"threads_started":1}
I20260806 15:02:19.010012 6685 leader_election.cc:302] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cccbf0ac00049ec9a9fb11e6bf8e7a7, 816466dc509349318085c9fbb52035ab; no voters:
I20260806 15:02:19.010355 7018 raft_consensus.cc:2781] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 15:02:19.010540 7018 raft_consensus.cc:482] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 15:02:19.010601 7018 raft_consensus.cc:3037] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:19.010895 6600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "717e8a75c20e46ac9c606af8da69b487" candidate_uuid: "816466dc509349318085c9fbb52035ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2974b69f8524717b7612e13f7149d55" is_pre_election: true
I20260806 15:02:19.011135 6600 raft_consensus.cc:2454] T 717e8a75c20e46ac9c606af8da69b487 P d2974b69f8524717b7612e13f7149d55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 816466dc509349318085c9fbb52035ab in term 0.
I20260806 15:02:19.012095 7018 raft_consensus.cc:504] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cccbf0ac00049ec9a9fb11e6bf8e7a7" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41473 } } peers { permanent_uuid: "d2974b69f8524717b7612e13f7149d55" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 32831 } } peers { permanent_uuid: "816466dc509349318085c9fbb52035ab" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41941 } }
I20260806 15:02:19.012334 7018 leader_election.cc:288] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab [CANDIDATE]: Term 1 election: Requested vote from peers 7cccbf0ac00049ec9a9fb11e6bf8e7a7 (127.11.8.67:41473), d2974b69f8524717b7612e13f7149d55 (127.11.8.65:32831)
I20260806 15:02:19.012598 6600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "717e8a75c20e46ac9c606af8da69b487" candidate_uuid: "816466dc509349318085c9fbb52035ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2974b69f8524717b7612e13f7149d55"
I20260806 15:02:19.012646 6940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "717e8a75c20e46ac9c606af8da69b487" candidate_uuid: "816466dc509349318085c9fbb52035ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cccbf0ac00049ec9a9fb11e6bf8e7a7"
I20260806 15:02:19.012712 6600 raft_consensus.cc:3037] T 717e8a75c20e46ac9c606af8da69b487 P d2974b69f8524717b7612e13f7149d55 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:19.012758 6940 raft_consensus.cc:3037] T 717e8a75c20e46ac9c606af8da69b487 P 7cccbf0ac00049ec9a9fb11e6bf8e7a7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:19.014276 6940 raft_consensus.cc:2454] T 717e8a75c20e46ac9c606af8da69b487 P 7cccbf0ac00049ec9a9fb11e6bf8e7a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 816466dc509349318085c9fbb52035ab in term 1.
I20260806 15:02:19.014276 6600 raft_consensus.cc:2454] T 717e8a75c20e46ac9c606af8da69b487 P d2974b69f8524717b7612e13f7149d55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 816466dc509349318085c9fbb52035ab in term 1.
I20260806 15:02:19.014720 6688 leader_election.cc:302] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 816466dc509349318085c9fbb52035ab, d2974b69f8524717b7612e13f7149d55; no voters:
I20260806 15:02:19.015120 7018 raft_consensus.cc:2781] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:19.015489 7018 raft_consensus.cc:686] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab [term 1 LEADER]: Becoming Leader. State: Replica: 816466dc509349318085c9fbb52035ab, State: Running, Role: LEADER
I20260806 15:02:19.015964 7018 consensus_queue.cc:237] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cccbf0ac00049ec9a9fb11e6bf8e7a7" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41473 } } peers { permanent_uuid: "d2974b69f8524717b7612e13f7149d55" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 32831 } } peers { permanent_uuid: "816466dc509349318085c9fbb52035ab" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41941 } }
I20260806 15:02:19.017379 6414 catalog_manager.cc:5495] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 816466dc509349318085c9fbb52035ab (127.11.8.66). New cstate: current_term: 1 leader_uuid: "816466dc509349318085c9fbb52035ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cccbf0ac00049ec9a9fb11e6bf8e7a7" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2974b69f8524717b7612e13f7149d55" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 32831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "816466dc509349318085c9fbb52035ab" member_type: VOTER last_known_addr { host: "127.11.8.66" port: 41941 } health_report { overall_health: HEALTHY } } }
W20260806 15:02:19.037417 6833 tablet.cc:2369] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 15:02:19.037982 7009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:19.034195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46624 (local address 127.11.8.66:41941)
0806 15:02:19.034296 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:19.034299 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:19.034308 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:02:19.034493 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:19.034494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:19.034527 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:19.034629 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:02:19.034846 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:19.035499 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:19.036706 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:19.036989 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:19.037073 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:19.037288 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:02:19.037466 (+ 178us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:02:19.037643 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:19.037765 (+ 122us) server_negotiation.cc:299] Negotiation successful
0806 15:02:19.037798 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 15:02:19.041015 6940 raft_consensus.cc:1261] T 717e8a75c20e46ac9c606af8da69b487 P 7cccbf0ac00049ec9a9fb11e6bf8e7a7 [term 1 FOLLOWER]: Refusing update from remote peer 816466dc509349318085c9fbb52035ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:02:19.041550 6600 raft_consensus.cc:1261] T 717e8a75c20e46ac9c606af8da69b487 P d2974b69f8524717b7612e13f7149d55 [term 1 FOLLOWER]: Refusing update from remote peer 816466dc509349318085c9fbb52035ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:02:19.041618 7021 consensus_queue.cc:1035] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cccbf0ac00049ec9a9fb11e6bf8e7a7" member_type: VOTER last_known_addr { host: "127.11.8.67" port: 41473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:02:19.042150 7018 consensus_queue.cc:1035] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2974b69f8524717b7612e13f7149d55" member_type: VOTER last_known_addr { host: "127.11.8.65" port: 32831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:02:19.046346 7027 mvcc.cc:205] Tried to move back new op lower bound from 7315572895907811328 to 7315572895810682880. Current Snapshot: MvccSnapshot[applied={T|T < 7315572895907811328}]
I20260806 15:02:19.048686 7030 mvcc.cc:205] Tried to move back new op lower bound from 7315572895907811328 to 7315572895810682880. Current Snapshot: MvccSnapshot[applied={T|T < 7315572895907811328 or (T in {7315572895907811328})}]
I20260806 15:02:19.051687 6414 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:02:19.051779 6414 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 15:02:19.052471 6414 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a584547a3b9f4ccd99ca73e87f6d3b15: Sending DeleteTablet for 3 replicas of tablet 717e8a75c20e46ac9c606af8da69b487
I20260806 15:02:19.052842 6577 tablet_service.cc:1514] Processing DeleteTablet for tablet 717e8a75c20e46ac9c606af8da69b487 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:02:19 UTC) from {username='slave'} at 127.0.0.1:48798
I20260806 15:02:19.052871 6916 tablet_service.cc:1514] Processing DeleteTablet for tablet 717e8a75c20e46ac9c606af8da69b487 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:02:19 UTC) from {username='slave'} at 127.0.0.1:41276
I20260806 15:02:19.053094 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 6493
I20260806 15:02:19.053078 6745 tablet_service.cc:1514] Processing DeleteTablet for tablet 717e8a75c20e46ac9c606af8da69b487 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 15:02:19 UTC) from {username='slave'} at 127.0.0.1:46612
I20260806 15:02:19.064368 7045 tablet_replica.cc:330] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab: stopping tablet replica
I20260806 15:02:19.064502 7045 raft_consensus.cc:2229] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab [term 1 LEADER]: Raft consensus shutting down.
I20260806 15:02:19.064628 7045 raft_consensus.cc:2258] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:02:19.064818 7043 tablet_replica.cc:330] T 717e8a75c20e46ac9c606af8da69b487 P 7cccbf0ac00049ec9a9fb11e6bf8e7a7: stopping tablet replica
I20260806 15:02:19.064930 7043 raft_consensus.cc:2229] T 717e8a75c20e46ac9c606af8da69b487 P 7cccbf0ac00049ec9a9fb11e6bf8e7a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 15:02:19.065063 7043 raft_consensus.cc:2258] T 717e8a75c20e46ac9c606af8da69b487 P 7cccbf0ac00049ec9a9fb11e6bf8e7a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 15:02:19.065145 7045 ts_tablet_manager.cc:1905] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:02:19.065624 7043 ts_tablet_manager.cc:1905] T 717e8a75c20e46ac9c606af8da69b487 P 7cccbf0ac00049ec9a9fb11e6bf8e7a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 15:02:19.067006 7045 ts_tablet_manager.cc:1918] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:02:19.067090 7045 log.cc:1196] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-1/wal/wals/717e8a75c20e46ac9c606af8da69b487
I20260806 15:02:19.067373 7045 ts_tablet_manager.cc:1939] T 717e8a75c20e46ac9c606af8da69b487 P 816466dc509349318085c9fbb52035ab: Deleting consensus metadata
I20260806 15:02:19.067698 7043 ts_tablet_manager.cc:1918] T 717e8a75c20e46ac9c606af8da69b487 P 7cccbf0ac00049ec9a9fb11e6bf8e7a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 15:02:19.067837 7043 log.cc:1196] T 717e8a75c20e46ac9c606af8da69b487 P 7cccbf0ac00049ec9a9fb11e6bf8e7a7: Deleting WAL directory at /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786028287744255-11297-0/minicluster-data/ts-2/wal/wals/717e8a75c20e46ac9c606af8da69b487
I20260806 15:02:19.067847 6396 catalog_manager.cc:4841] TS 816466dc509349318085c9fbb52035ab (127.11.8.66:41941): tablet 717e8a75c20e46ac9c606af8da69b487 (table test-table [id=84f57bbec02048f2bb9aada582d54da6]) successfully deleted
I20260806 15:02:19.068171 7043 ts_tablet_manager.cc:1939] T 717e8a75c20e46ac9c606af8da69b487 P 7cccbf0ac00049ec9a9fb11e6bf8e7a7: Deleting consensus metadata
I20260806 15:02:19.068564 6396 catalog_manager.cc:4841] TS 7cccbf0ac00049ec9a9fb11e6bf8e7a7 (127.11.8.67:41473): tablet 717e8a75c20e46ac9c606af8da69b487 (table test-table [id=84f57bbec02048f2bb9aada582d54da6]) successfully deleted
W20260806 15:02:19.070263 6401 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.11.8.65:32831 (error 108)
I20260806 15:02:19.070791 7005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:19.070658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.65:32831 (local address 127.0.0.1:48808)
0806 15:02:19.070710 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:19.070754 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.65:32831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 15:02:19.070884 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 6665
W20260806 15:02:19.074319 6401 catalog_manager.cc:4584] TS d2974b69f8524717b7612e13f7149d55 (127.11.8.65:32831): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 717e8a75c20e46ac9c606af8da69b487: Network error: Client connection negotiation failed: client connection to 127.11.8.65:32831: connect: Connection refused (error 111)
I20260806 15:02:19.079393 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 6835
I20260806 15:02:19.086447 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 6376
2026-08-06T15:02:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (780 ms)
[----------] 6 tests from EncryptionPolicyTest (9180 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:02:19 dist-test-slave-68xk krb5kdc[7049](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:02:19 dist-test-slave-68xk krb5kdc[7049](info): set up 2 sockets
Aug 06 15:02:19 dist-test-slave-68xk krb5kdc[7049](info): commencing operation
krb5kdc: starting...
W20260806 15:02:20.280433 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 15:02:20 dist-test-slave-68xk krb5kdc[7049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:02:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:02:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 15:02:20.405440 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37275
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:20.488641 7066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:20.488837 7066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:20.488860 7066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:20.489969 7066 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:20.490010 7066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:20.490025 7066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:20.490038 7066 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:20.490051 7066 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:20.491585 7066 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37275
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:20.491837 7066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:20.492067 7066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:20.497021 7074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:20.497001 7071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:20.497311 7066 server_base.cc:1034] running on GCE node
W20260806 15:02:20.496997 7072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:20.497601 7066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:20.497877 7066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:20.499040 7066 hybrid_clock.cc:648] HybridClock initialized: now 1786028540499078 us; error 99 us; skew 500 ppm
Aug 06 15:02:20 dist-test-slave-68xk krb5kdc[7049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028540, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:20.501506 7066 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 15:02:20.501967 7066 webserver.cc:460] Webserver started at http://127.0.0.1:36385/ using document root <none> and password file <none>
I20260806 15:02:20.502197 7066 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:20.502262 7066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:20.502446 7066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:20.503513 7066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "1c2a1e8d890847afad80063ce9bc0f44"
format_stamp: "Formatted at 2026-08-06 15:02:20 on dist-test-slave-68xk"
server_key: "68b7ef7a9c5bd4d89b557dc5f9a066e0"
server_key_iv: "e369f1063059d694e6b4cccfcbd4f7df"
server_key_version: "encryptionkey@0"
I20260806 15:02:20.503880 7066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "1c2a1e8d890847afad80063ce9bc0f44"
format_stamp: "Formatted at 2026-08-06 15:02:20 on dist-test-slave-68xk"
server_key: "68b7ef7a9c5bd4d89b557dc5f9a066e0"
server_key_iv: "e369f1063059d694e6b4cccfcbd4f7df"
server_key_version: "encryptionkey@0"
I20260806 15:02:20.505273 7066 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:02:20.506023 7082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:20.506240 7066 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:02:20.506340 7066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "1c2a1e8d890847afad80063ce9bc0f44"
format_stamp: "Formatted at 2026-08-06 15:02:20 on dist-test-slave-68xk"
server_key: "68b7ef7a9c5bd4d89b557dc5f9a066e0"
server_key_iv: "e369f1063059d694e6b4cccfcbd4f7df"
server_key_version: "encryptionkey@0"
I20260806 15:02:20.506443 7066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:20.519429 7066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:20.521633 7066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:20.521795 7066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:20.525990 7066 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37275
I20260806 15:02:20.526036 7150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37275 every 8 connection(s)
I20260806 15:02:20.526379 7066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:02:20.527279 7151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:20.529503 7151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c2a1e8d890847afad80063ce9bc0f44: Bootstrap starting.
I20260806 15:02:20.530055 7151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c2a1e8d890847afad80063ce9bc0f44: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:20.530303 7151 log.cc:824] T 00000000000000000000000000000000 P 1c2a1e8d890847afad80063ce9bc0f44: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:20.530763 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 7066
I20260806 15:02:20.530900 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:02:20.531149 11297 external_mini_cluster.cc:1428] Setting key 429dc550b671fef2b17f57efd38a4cca
I20260806 15:02:20.531272 7151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c2a1e8d890847afad80063ce9bc0f44: No bootstrap required, opened a new log
I20260806 15:02:20.533531 7151 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c2a1e8d890847afad80063ce9bc0f44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2a1e8d890847afad80063ce9bc0f44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37275 } }
I20260806 15:02:20.533718 7151 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c2a1e8d890847afad80063ce9bc0f44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:20.533759 7151 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c2a1e8d890847afad80063ce9bc0f44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c2a1e8d890847afad80063ce9bc0f44, State: Initialized, Role: FOLLOWER
I20260806 15:02:20.533910 7151 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c2a1e8d890847afad80063ce9bc0f44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2a1e8d890847afad80063ce9bc0f44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37275 } }
I20260806 15:02:20.533991 7151 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c2a1e8d890847afad80063ce9bc0f44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:02:20.534029 7151 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c2a1e8d890847afad80063ce9bc0f44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:02:20.534073 7151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c2a1e8d890847afad80063ce9bc0f44 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:20.534814 7151 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c2a1e8d890847afad80063ce9bc0f44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2a1e8d890847afad80063ce9bc0f44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37275 } }
I20260806 15:02:20.534930 7151 leader_election.cc:302] T 00000000000000000000000000000000 P 1c2a1e8d890847afad80063ce9bc0f44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c2a1e8d890847afad80063ce9bc0f44; no voters:
I20260806 15:02:20.535086 7151 leader_election.cc:288] T 00000000000000000000000000000000 P 1c2a1e8d890847afad80063ce9bc0f44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:02:20.535231 7159 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c2a1e8d890847afad80063ce9bc0f44 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:20.535308 7151 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c2a1e8d890847afad80063ce9bc0f44 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:20.535379 7159 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c2a1e8d890847afad80063ce9bc0f44 [term 1 LEADER]: Becoming Leader. State: Replica: 1c2a1e8d890847afad80063ce9bc0f44, State: Running, Role: LEADER
I20260806 15:02:20.535488 7159 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c2a1e8d890847afad80063ce9bc0f44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2a1e8d890847afad80063ce9bc0f44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37275 } }
I20260806 15:02:20.535825 7159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c2a1e8d890847afad80063ce9bc0f44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c2a1e8d890847afad80063ce9bc0f44. Latest consensus state: current_term: 1 leader_uuid: "1c2a1e8d890847afad80063ce9bc0f44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2a1e8d890847afad80063ce9bc0f44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37275 } } }
I20260806 15:02:20.535887 7159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c2a1e8d890847afad80063ce9bc0f44 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:20.536044 7160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c2a1e8d890847afad80063ce9bc0f44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c2a1e8d890847afad80063ce9bc0f44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c2a1e8d890847afad80063ce9bc0f44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37275 } } }
I20260806 15:02:20.536115 7160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c2a1e8d890847afad80063ce9bc0f44 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:20.536325 7165 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:02:20.536837 7165 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:02:20.538547 7165 catalog_manager.cc:1347] Generated new cluster ID: 3bafb31faf644662812afe452229814c
I20260806 15:02:20.538609 7165 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 15:02:20 dist-test-slave-68xk krb5kdc[7049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 15:02:20.543962 7157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:20.531688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53683 (local address 127.0.0.1:37275)
0806 15:02:20.531883 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:20.531893 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:20.531927 (+ 34us) server_negotiation.cc:407] Connection header received
0806 15:02:20.532700 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:20.532710 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:20.532771 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:20.532917 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:20.533541 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:20.535282 (+ 1741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:20.536444 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:20.537202 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:20.537323 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:20.540296 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:20.540301 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:20.540301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:20.540312 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:20.542191 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:20.542681 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:20.542685 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:20.542685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:20.542795 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:20.543043 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:20.543044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:20.543044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:20.543322 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:20.543476 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:20.543637 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 15:02:20.543739 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":95,"threads_started":1}
I20260806 15:02:20.545328 11297 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:02:20.551434 7165 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:02:20.552037 7165 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:02:20.554085 7157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:20.546807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48203 (local address 127.0.0.1:37275)
0806 15:02:20.546838 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:20.546840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:20.546938 (+ 98us) server_negotiation.cc:407] Connection header received
0806 15:02:20.546987 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:20.546988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:20.546996 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:20.547052 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:20.547656 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:20.548360 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:20.549358 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:20.549679 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:20.549750 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:20.551142 (+ 1392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:20.551144 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:20.551144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:20.551151 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:20.552743 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:20.553156 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:20.553157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:20.553157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:20.553216 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:20.553444 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:20.553445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:20.553445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:20.553570 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:20.553668 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:20.553821 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 15:02:20.553886 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 15:02:20.555154 7165 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c2a1e8d890847afad80063ce9bc0f44: Generated new TSK 0
I20260806 15:02:20.555333 7165 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:02:20.558641 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 7066
W20260806 15:02:20.571559 7047 connection.cc:535] client connection to 127.0.0.1:37275 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37275): Connection reset by peer (error 104)
2026-08-06T15:02:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1476 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:02:20 dist-test-slave-68xk krb5kdc[7193](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:02:20 dist-test-slave-68xk krb5kdc[7193](info): set up 2 sockets
Aug 06 15:02:20 dist-test-slave-68xk krb5kdc[7193](info): commencing operation
krb5kdc: starting...
W20260806 15:02:21.764791 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 15:02:21 dist-test-slave-68xk krb5kdc[7193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:02:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:02:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 15:02:21.888893 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35637
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:21.970932 7208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:21.971133 7208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:21.971155 7208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:21.972295 7208 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:21.972340 7208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:21.972355 7208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:21.972368 7208 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:21.972380 7208 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:21.973896 7208 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35637
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:21.974148 7208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:21.974392 7208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:21.979385 7216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:21.979419 7218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:21.979437 7215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:21.979756 7208 server_base.cc:1034] running on GCE node
I20260806 15:02:21.980028 7208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:21.980337 7208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:21.981518 7208 hybrid_clock.cc:648] HybridClock initialized: now 1786028541981494 us; error 37 us; skew 500 ppm
Aug 06 15:02:21 dist-test-slave-68xk krb5kdc[7193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028541, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:21.984258 7208 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 15:02:21.984771 7208 webserver.cc:460] Webserver started at http://127.0.0.1:40945/ using document root <none> and password file <none>
I20260806 15:02:21.985040 7208 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:21.985090 7208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:21.985260 7208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:21.986354 7208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "7cf4c1141ac841ae9b765b354c8d1b74"
format_stamp: "Formatted at 2026-08-06 15:02:21 on dist-test-slave-68xk"
server_key: "b01d4f967461631e9b70fb905a93d978"
server_key_iv: "5a0aa4512cb976a6b4e66881aa6dff90"
server_key_version: "encryptionkey@0"
I20260806 15:02:21.986773 7208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "7cf4c1141ac841ae9b765b354c8d1b74"
format_stamp: "Formatted at 2026-08-06 15:02:21 on dist-test-slave-68xk"
server_key: "b01d4f967461631e9b70fb905a93d978"
server_key_iv: "5a0aa4512cb976a6b4e66881aa6dff90"
server_key_version: "encryptionkey@0"
I20260806 15:02:21.988188 7208 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:02:21.989025 7228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:21.989259 7208 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:02:21.989363 7208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "7cf4c1141ac841ae9b765b354c8d1b74"
format_stamp: "Formatted at 2026-08-06 15:02:21 on dist-test-slave-68xk"
server_key: "b01d4f967461631e9b70fb905a93d978"
server_key_iv: "5a0aa4512cb976a6b4e66881aa6dff90"
server_key_version: "encryptionkey@0"
I20260806 15:02:21.989437 7208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:22.003423 7208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:22.005988 7208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:22.006219 7208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:22.010911 7208 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35637
I20260806 15:02:22.010962 7299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35637 every 8 connection(s)
I20260806 15:02:22.011363 7208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:02:22.012363 7300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:22.013953 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 7208
I20260806 15:02:22.014081 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:02:22.014333 11297 external_mini_cluster.cc:1428] Setting key 9a3765bc5e4b4934b15ad1ba70b9f352
I20260806 15:02:22.015484 7300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cf4c1141ac841ae9b765b354c8d1b74: Bootstrap starting.
I20260806 15:02:22.016327 7300 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cf4c1141ac841ae9b765b354c8d1b74: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:22.016721 7300 log.cc:824] T 00000000000000000000000000000000 P 7cf4c1141ac841ae9b765b354c8d1b74: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:22.017616 7300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cf4c1141ac841ae9b765b354c8d1b74: No bootstrap required, opened a new log
I20260806 15:02:22.020144 7300 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cf4c1141ac841ae9b765b354c8d1b74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf4c1141ac841ae9b765b354c8d1b74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35637 } }
I20260806 15:02:22.020365 7300 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7cf4c1141ac841ae9b765b354c8d1b74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:22.020412 7300 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cf4c1141ac841ae9b765b354c8d1b74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cf4c1141ac841ae9b765b354c8d1b74, State: Initialized, Role: FOLLOWER
I20260806 15:02:22.020565 7300 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cf4c1141ac841ae9b765b354c8d1b74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf4c1141ac841ae9b765b354c8d1b74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35637 } }
I20260806 15:02:22.020684 7300 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7cf4c1141ac841ae9b765b354c8d1b74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:02:22.020733 7300 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cf4c1141ac841ae9b765b354c8d1b74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:02:22.020792 7300 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cf4c1141ac841ae9b765b354c8d1b74 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:22.021087 7305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:22.014987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42043 (local address 127.0.0.1:35637)
0806 15:02:22.015218 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:22.015229 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:22.015265 (+ 36us) server_negotiation.cc:407] Connection header received
0806 15:02:22.015861 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:22.015872 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:22.015948 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:22.016117 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:22.016414 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:22.017853 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:22.018712 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:22.019419 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:22.019582 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:22.020250 (+ 668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:22.020255 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:22.020255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:22.020483 (+ 228us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:22.020531 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:22.020548 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:22.020647 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:22.020810 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 15:02:22.020842 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":96,"threads_started":1}
I20260806 15:02:22.021555 7300 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cf4c1141ac841ae9b765b354c8d1b74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf4c1141ac841ae9b765b354c8d1b74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35637 } }
I20260806 15:02:22.021680 7300 leader_election.cc:302] T 00000000000000000000000000000000 P 7cf4c1141ac841ae9b765b354c8d1b74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cf4c1141ac841ae9b765b354c8d1b74; no voters:
I20260806 15:02:22.021883 7300 leader_election.cc:288] T 00000000000000000000000000000000 P 7cf4c1141ac841ae9b765b354c8d1b74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:02:22.021903 7308 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cf4c1141ac841ae9b765b354c8d1b74 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:22.022027 7308 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7cf4c1141ac841ae9b765b354c8d1b74 [term 1 LEADER]: Becoming Leader. State: Replica: 7cf4c1141ac841ae9b765b354c8d1b74, State: Running, Role: LEADER
I20260806 15:02:22.022117 7308 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cf4c1141ac841ae9b765b354c8d1b74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf4c1141ac841ae9b765b354c8d1b74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35637 } }
I20260806 15:02:22.022156 7300 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cf4c1141ac841ae9b765b354c8d1b74 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:22.022429 11297 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:02:22.022504 7310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cf4c1141ac841ae9b765b354c8d1b74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cf4c1141ac841ae9b765b354c8d1b74. Latest consensus state: current_term: 1 leader_uuid: "7cf4c1141ac841ae9b765b354c8d1b74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf4c1141ac841ae9b765b354c8d1b74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35637 } } }
I20260806 15:02:22.022580 7310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cf4c1141ac841ae9b765b354c8d1b74 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:22.022765 7309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cf4c1141ac841ae9b765b354c8d1b74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7cf4c1141ac841ae9b765b354c8d1b74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cf4c1141ac841ae9b765b354c8d1b74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35637 } } }
I20260806 15:02:22.022830 7309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cf4c1141ac841ae9b765b354c8d1b74 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:22.023208 7314 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:02:22.023766 7314 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:02:22.025516 7314 catalog_manager.cc:1347] Generated new cluster ID: 5c991e6ae43748f7acb44b86318cd000
I20260806 15:02:22.025575 7314 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:02:22.027509 7305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:22.024166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33917 (local address 127.0.0.1:35637)
0806 15:02:22.024205 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:22.024207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:22.024406 (+ 199us) server_negotiation.cc:407] Connection header received
0806 15:02:22.024513 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:22.024514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:22.024524 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:22.024553 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:22.024740 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:22.025419 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:22.026206 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:22.026487 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:22.026572 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:22.027265 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:22.027266 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:22.027267 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:22.027271 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:22.027286 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:22.027290 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:22.027344 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:22.027406 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 15:02:22.027415 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 15:02:22.042717 7314 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:02:22.043253 7314 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:02:22.059531 7314 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7cf4c1141ac841ae9b765b354c8d1b74: Generated new TSK 0
I20260806 15:02:22.059749 7314 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:02:22.064181 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 7208
W20260806 15:02:22.080255 7190 connection.cc:535] client connection to 127.0.0.1:35637 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35637): Connection reset by peer (error 104)
2026-08-06T15:02:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1508 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:02:22 dist-test-slave-68xk krb5kdc[7345](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:02:22 dist-test-slave-68xk krb5kdc[7345](info): set up 2 sockets
Aug 06 15:02:22 dist-test-slave-68xk krb5kdc[7345](info): commencing operation
krb5kdc: starting...
W20260806 15:02:23.324306 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 15:02:23 dist-test-slave-68xk krb5kdc[7345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:02:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:02:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 15:02:23.453879 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35761
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:23.539333 7364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:23.539588 7364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:23.539628 7364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:23.540772 7364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:23.540839 7364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:23.540864 7364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:23.540894 7364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:23.540922 7364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:23.542444 7364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35761
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:23.542727 7364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:23.542953 7364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:23.548043 7369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:23.548044 7374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:23.548066 7364 server_base.cc:1034] running on GCE node
W20260806 15:02:23.548075 7370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:23.548521 7364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:23.548851 7364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:23.550038 7364 hybrid_clock.cc:648] HybridClock initialized: now 1786028543550026 us; error 38 us; skew 500 ppm
Aug 06 15:02:23 dist-test-slave-68xk krb5kdc[7345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028543, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:23.558650 7364 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 15:02:23.559192 7364 webserver.cc:460] Webserver started at http://127.0.0.1:42699/ using document root <none> and password file <none>
I20260806 15:02:23.559448 7364 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:23.559513 7364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:23.559648 7364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:23.560753 7364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "0c6728e2c1664dee9b98ace583b9994e"
format_stamp: "Formatted at 2026-08-06 15:02:23 on dist-test-slave-68xk"
server_key: "c5d38db95328da36bdbe9a32fa60d7c8"
server_key_iv: "75a17ec1bf18a514de4cb3da6f6f14a5"
server_key_version: "encryptionkey@0"
I20260806 15:02:23.561122 7364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "0c6728e2c1664dee9b98ace583b9994e"
format_stamp: "Formatted at 2026-08-06 15:02:23 on dist-test-slave-68xk"
server_key: "c5d38db95328da36bdbe9a32fa60d7c8"
server_key_iv: "75a17ec1bf18a514de4cb3da6f6f14a5"
server_key_version: "encryptionkey@0"
I20260806 15:02:23.562546 7364 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:02:23.563310 7383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:23.563488 7364 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:02:23.563585 7364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "0c6728e2c1664dee9b98ace583b9994e"
format_stamp: "Formatted at 2026-08-06 15:02:23 on dist-test-slave-68xk"
server_key: "c5d38db95328da36bdbe9a32fa60d7c8"
server_key_iv: "75a17ec1bf18a514de4cb3da6f6f14a5"
server_key_version: "encryptionkey@0"
I20260806 15:02:23.563668 7364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:23.567469 7364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:23.569586 7364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:23.569738 7364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:23.573733 7364 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35761
I20260806 15:02:23.573765 7450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35761 every 8 connection(s)
I20260806 15:02:23.574100 7364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:02:23.575153 7451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:23.577622 7451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c6728e2c1664dee9b98ace583b9994e: Bootstrap starting.
I20260806 15:02:23.578255 7451 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c6728e2c1664dee9b98ace583b9994e: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:23.578631 7451 log.cc:824] T 00000000000000000000000000000000 P 0c6728e2c1664dee9b98ace583b9994e: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:23.579388 7451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c6728e2c1664dee9b98ace583b9994e: No bootstrap required, opened a new log
I20260806 15:02:23.579387 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 7364
I20260806 15:02:23.579542 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:02:23.579759 11297 external_mini_cluster.cc:1428] Setting key eff9a7937902f01c9794b018d04afde2
I20260806 15:02:23.581425 7451 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c6728e2c1664dee9b98ace583b9994e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c6728e2c1664dee9b98ace583b9994e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35761 } }
I20260806 15:02:23.581583 7451 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c6728e2c1664dee9b98ace583b9994e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:23.581666 7451 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c6728e2c1664dee9b98ace583b9994e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c6728e2c1664dee9b98ace583b9994e, State: Initialized, Role: FOLLOWER
I20260806 15:02:23.581830 7451 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c6728e2c1664dee9b98ace583b9994e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c6728e2c1664dee9b98ace583b9994e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35761 } }
I20260806 15:02:23.581934 7451 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c6728e2c1664dee9b98ace583b9994e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:02:23.581971 7451 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c6728e2c1664dee9b98ace583b9994e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:02:23.582072 7451 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c6728e2c1664dee9b98ace583b9994e [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:23.582908 7451 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c6728e2c1664dee9b98ace583b9994e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c6728e2c1664dee9b98ace583b9994e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35761 } }
I20260806 15:02:23.583050 7451 leader_election.cc:302] T 00000000000000000000000000000000 P 0c6728e2c1664dee9b98ace583b9994e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c6728e2c1664dee9b98ace583b9994e; no voters:
I20260806 15:02:23.583155 7459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:23.580233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42975 (local address 127.0.0.1:35761)
0806 15:02:23.580422 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:23.580432 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:23.580465 (+ 33us) server_negotiation.cc:407] Connection header received
0806 15:02:23.581597 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:23.581633 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:23.581674 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:23.581832 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:23.582641 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:23.582645 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:23.582645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:23.582649 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:23.582688 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:23.582698 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:23.582755 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:23.582845 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 15:02:23.582896 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":82,"threads_started":1}
I20260806 15:02:23.583241 7451 leader_election.cc:288] T 00000000000000000000000000000000 P 0c6728e2c1664dee9b98ace583b9994e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:02:23.583425 7461 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c6728e2c1664dee9b98ace583b9994e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:23.583585 7461 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c6728e2c1664dee9b98ace583b9994e [term 1 LEADER]: Becoming Leader. State: Replica: 0c6728e2c1664dee9b98ace583b9994e, State: Running, Role: LEADER
I20260806 15:02:23.583678 7451 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c6728e2c1664dee9b98ace583b9994e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:23.583763 7461 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c6728e2c1664dee9b98ace583b9994e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c6728e2c1664dee9b98ace583b9994e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35761 } }
I20260806 15:02:23.584161 11297 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:02:23.584164 7460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c6728e2c1664dee9b98ace583b9994e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c6728e2c1664dee9b98ace583b9994e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c6728e2c1664dee9b98ace583b9994e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35761 } } }
I20260806 15:02:23.584249 7460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c6728e2c1664dee9b98ace583b9994e [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:23.584481 7466 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:02:23.584834 7462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c6728e2c1664dee9b98ace583b9994e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c6728e2c1664dee9b98ace583b9994e. Latest consensus state: current_term: 1 leader_uuid: "0c6728e2c1664dee9b98ace583b9994e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c6728e2c1664dee9b98ace583b9994e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35761 } } }
I20260806 15:02:23.585063 7466 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:02:23.585317 7462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c6728e2c1664dee9b98ace583b9994e [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:23.587392 7466 catalog_manager.cc:1347] Generated new cluster ID: c38485479fc14f63b90bbaecff254d4d
I20260806 15:02:23.587493 7466 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:02:23.587739 7459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:23.586052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42977 (local address 127.0.0.1:35761)
0806 15:02:23.586096 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:23.586098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:23.586633 (+ 535us) server_negotiation.cc:407] Connection header received
0806 15:02:23.586736 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:23.586738 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:23.586758 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:23.586835 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:23.587447 (+ 612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:23.587448 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:23.587449 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:23.587453 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:23.587467 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:23.587471 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:23.587520 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:23.587632 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 15:02:23.587648 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 15:02:23.595126 7466 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:02:23.595331 7466 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:02:23.600876 7466 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c6728e2c1664dee9b98ace583b9994e: Generated new TSK 0
I20260806 15:02:23.601065 7466 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:02:23.606722 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 7364
2026-08-06T15:02:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1544 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:02:23 dist-test-slave-68xk krb5kdc[7488](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:02:23 dist-test-slave-68xk krb5kdc[7488](info): set up 2 sockets
Aug 06 15:02:23 dist-test-slave-68xk krb5kdc[7488](info): commencing operation
krb5kdc: starting...
W20260806 15:02:24.832208 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 15:02:24 dist-test-slave-68xk krb5kdc[7488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:02:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:02:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 15:02:24.958981 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44975
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:25.045552 7505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:25.045756 7505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:25.045779 7505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:25.046937 7505 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:25.046979 7505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:25.046994 7505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:25.047006 7505 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:25.047041 7505 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:25.048548 7505 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44975
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:25.048802 7505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:25.049023 7505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:25.054023 7511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:25.054050 7512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:25.054010 7514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:25.054374 7505 server_base.cc:1034] running on GCE node
I20260806 15:02:25.054646 7505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:25.054981 7505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:25.056159 7505 hybrid_clock.cc:648] HybridClock initialized: now 1786028545056140 us; error 38 us; skew 500 ppm
Aug 06 15:02:25 dist-test-slave-68xk krb5kdc[7488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028545, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:25.058665 7505 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 15:02:25.059150 7505 webserver.cc:460] Webserver started at http://127.0.0.1:34143/ using document root <none> and password file <none>
I20260806 15:02:25.059381 7505 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:25.059448 7505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:25.059602 7505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:25.060642 7505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "4edcae20b86a4eff86dc19737f4bdb95"
format_stamp: "Formatted at 2026-08-06 15:02:25 on dist-test-slave-68xk"
server_key: "484211d9bb06273908f0a34719352eae"
server_key_iv: "9cd5240c2f0d9aa50495f86d46c2192c"
server_key_version: "encryptionkey@0"
I20260806 15:02:25.061005 7505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "4edcae20b86a4eff86dc19737f4bdb95"
format_stamp: "Formatted at 2026-08-06 15:02:25 on dist-test-slave-68xk"
server_key: "484211d9bb06273908f0a34719352eae"
server_key_iv: "9cd5240c2f0d9aa50495f86d46c2192c"
server_key_version: "encryptionkey@0"
I20260806 15:02:25.062407 7505 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:02:25.063263 7523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:25.063441 7505 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:02:25.063541 7505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "4edcae20b86a4eff86dc19737f4bdb95"
format_stamp: "Formatted at 2026-08-06 15:02:25 on dist-test-slave-68xk"
server_key: "484211d9bb06273908f0a34719352eae"
server_key_iv: "9cd5240c2f0d9aa50495f86d46c2192c"
server_key_version: "encryptionkey@0"
I20260806 15:02:25.063627 7505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:25.091662 7505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:25.093694 7505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:25.093858 7505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:25.098026 7505 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44975
I20260806 15:02:25.098068 7588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44975 every 8 connection(s)
I20260806 15:02:25.098517 7505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:02:25.099398 7589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:25.101734 7589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4edcae20b86a4eff86dc19737f4bdb95: Bootstrap starting.
I20260806 15:02:25.102341 7589 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4edcae20b86a4eff86dc19737f4bdb95: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:25.102679 7589 log.cc:824] T 00000000000000000000000000000000 P 4edcae20b86a4eff86dc19737f4bdb95: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:25.103413 7589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4edcae20b86a4eff86dc19737f4bdb95: No bootstrap required, opened a new log
I20260806 15:02:25.104423 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 7505
I20260806 15:02:25.104543 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:02:25.104743 11297 external_mini_cluster.cc:1428] Setting key 62683bf3912c0d1322da896d331f0484
I20260806 15:02:25.105675 7589 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4edcae20b86a4eff86dc19737f4bdb95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4edcae20b86a4eff86dc19737f4bdb95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44975 } }
I20260806 15:02:25.105851 7589 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4edcae20b86a4eff86dc19737f4bdb95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:25.105892 7589 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4edcae20b86a4eff86dc19737f4bdb95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4edcae20b86a4eff86dc19737f4bdb95, State: Initialized, Role: FOLLOWER
I20260806 15:02:25.106067 7589 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4edcae20b86a4eff86dc19737f4bdb95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4edcae20b86a4eff86dc19737f4bdb95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44975 } }
I20260806 15:02:25.106164 7589 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4edcae20b86a4eff86dc19737f4bdb95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:02:25.106221 7589 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4edcae20b86a4eff86dc19737f4bdb95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:02:25.106278 7589 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4edcae20b86a4eff86dc19737f4bdb95 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:25.107311 7589 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4edcae20b86a4eff86dc19737f4bdb95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4edcae20b86a4eff86dc19737f4bdb95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44975 } }
I20260806 15:02:25.107401 7589 leader_election.cc:302] T 00000000000000000000000000000000 P 4edcae20b86a4eff86dc19737f4bdb95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4edcae20b86a4eff86dc19737f4bdb95; no voters:
I20260806 15:02:25.107585 7589 leader_election.cc:288] T 00000000000000000000000000000000 P 4edcae20b86a4eff86dc19737f4bdb95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:02:25.107610 7596 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4edcae20b86a4eff86dc19737f4bdb95 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:25.107730 7596 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4edcae20b86a4eff86dc19737f4bdb95 [term 1 LEADER]: Becoming Leader. State: Replica: 4edcae20b86a4eff86dc19737f4bdb95, State: Running, Role: LEADER
I20260806 15:02:25.107862 7589 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4edcae20b86a4eff86dc19737f4bdb95 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:25.107867 7596 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4edcae20b86a4eff86dc19737f4bdb95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4edcae20b86a4eff86dc19737f4bdb95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44975 } }
I20260806 15:02:25.108323 7597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4edcae20b86a4eff86dc19737f4bdb95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4edcae20b86a4eff86dc19737f4bdb95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4edcae20b86a4eff86dc19737f4bdb95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44975 } } }
I20260806 15:02:25.108335 7598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4edcae20b86a4eff86dc19737f4bdb95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4edcae20b86a4eff86dc19737f4bdb95. Latest consensus state: current_term: 1 leader_uuid: "4edcae20b86a4eff86dc19737f4bdb95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4edcae20b86a4eff86dc19737f4bdb95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44975 } } }
I20260806 15:02:25.108536 7598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4edcae20b86a4eff86dc19737f4bdb95 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:25.108745 7608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:02:25.108445 7597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4edcae20b86a4eff86dc19737f4bdb95 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:25.109308 7608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 15:02:25 dist-test-slave-68xk krb5kdc[7488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 15:02:25.114928 7608 catalog_manager.cc:1347] Generated new cluster ID: d5430ef16ac641e6850b63ebda8de271
I20260806 15:02:25.115001 7608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:02:25.116143 7594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:25.105310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42013 (local address 127.0.0.1:44975)
0806 15:02:25.105513 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:25.105521 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:25.105548 (+ 27us) server_negotiation.cc:407] Connection header received
0806 15:02:25.105651 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:25.105656 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:25.105708 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:25.105817 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:25.106710 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:25.107811 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:25.108975 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:25.109673 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:25.110121 (+ 448us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:25.112550 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:25.112554 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:25.112554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:25.112565 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:25.114232 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:25.114652 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:25.114655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:25.114655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:25.114800 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:25.115036 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:25.115038 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:25.115038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:25.115374 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:25.115545 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:25.115634 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 15:02:25.115777 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":109,"threads_started":1}
I20260806 15:02:25.117110 11297 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:02:25.129487 7594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:25.118636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53577 (local address 127.0.0.1:44975)
0806 15:02:25.118668 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:25.118670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:25.122604 (+ 3934us) server_negotiation.cc:407] Connection header received
0806 15:02:25.122667 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:25.122668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:25.122678 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:25.122734 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:25.123351 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:25.124058 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:25.125094 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:25.125405 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:25.125460 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:25.126832 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:25.126834 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:25.126834 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:25.126841 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:25.128363 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:25.128743 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:25.128744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:25.128744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:25.128797 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:25.128997 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:25.128997 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:25.128997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:25.129115 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:25.129200 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:25.129329 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 15:02:25.129400 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 15:02:25.131510 7608 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:02:25.132238 7608 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:02:25.138974 7608 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4edcae20b86a4eff86dc19737f4bdb95: Generated new TSK 0
I20260806 15:02:25.139197 7608 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:02:25.147922 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 7505
2026-08-06T15:02:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1538 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:02:25 dist-test-slave-68xk krb5kdc[7629](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:02:25 dist-test-slave-68xk krb5kdc[7629](info): set up 2 sockets
Aug 06 15:02:25 dist-test-slave-68xk krb5kdc[7629](info): commencing operation
krb5kdc: starting...
W20260806 15:02:26.354771 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 15:02:26 dist-test-slave-68xk krb5kdc[7629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:02:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:02:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 15:02:26.475343 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44939
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44939
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:26.561131 7646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:26.561352 7646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:26.561378 7646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:26.562574 7646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:26.562618 7646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:26.562633 7646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:26.562645 7646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:26.562659 7646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:26.564123 7646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:26.564327 7646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:26.564498 7646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:26.569334 7654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:26.569435 7657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:26.569558 7646 server_base.cc:1034] running on GCE node
W20260806 15:02:26.569435 7660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:26.569869 7646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:26.570254 7646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:26.571410 7646 hybrid_clock.cc:648] HybridClock initialized: now 1786028546571400 us; error 38 us; skew 500 ppm
Aug 06 15:02:26 dist-test-slave-68xk krb5kdc[7629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028546, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:26.573906 7646 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 15:02:26.574455 7646 webserver.cc:460] Webserver started at http://127.0.0.1:36849/ using document root <none> and password file <none>
I20260806 15:02:26.574692 7646 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:26.574757 7646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:26.574908 7646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:26.575930 7646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "7a9c5995cd6a4c61b4f0fdb4844208a7"
format_stamp: "Formatted at 2026-08-06 15:02:26 on dist-test-slave-68xk"
server_key: "345a6436d9f846c5a3c04e409f984431"
server_key_iv: "976b3ca5be97841aa50ceceddd6b0c8d"
server_key_version: "encryptionkey@0"
I20260806 15:02:26.576306 7646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "7a9c5995cd6a4c61b4f0fdb4844208a7"
format_stamp: "Formatted at 2026-08-06 15:02:26 on dist-test-slave-68xk"
server_key: "345a6436d9f846c5a3c04e409f984431"
server_key_iv: "976b3ca5be97841aa50ceceddd6b0c8d"
server_key_version: "encryptionkey@0"
I20260806 15:02:26.577659 7646 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:02:26.578380 7669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:26.578560 7646 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:02:26.578658 7646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "7a9c5995cd6a4c61b4f0fdb4844208a7"
format_stamp: "Formatted at 2026-08-06 15:02:26 on dist-test-slave-68xk"
server_key: "345a6436d9f846c5a3c04e409f984431"
server_key_iv: "976b3ca5be97841aa50ceceddd6b0c8d"
server_key_version: "encryptionkey@0"
I20260806 15:02:26.578733 7646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:26.588771 7646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:26.591041 7646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:26.591202 7646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:26.595317 7646 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44939
I20260806 15:02:26.595351 7735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44939 every 8 connection(s)
I20260806 15:02:26.595701 7646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:02:26.596549 7737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:26.598815 7737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a9c5995cd6a4c61b4f0fdb4844208a7: Bootstrap starting.
I20260806 15:02:26.599332 7737 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a9c5995cd6a4c61b4f0fdb4844208a7: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:26.599574 7737 log.cc:824] T 00000000000000000000000000000000 P 7a9c5995cd6a4c61b4f0fdb4844208a7: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:26.600212 7737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a9c5995cd6a4c61b4f0fdb4844208a7: No bootstrap required, opened a new log
I20260806 15:02:26.600832 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 7646
I20260806 15:02:26.600952 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:02:26.601178 11297 external_mini_cluster.cc:1428] Setting key 1e704e1cf3d26cef89ea646ab5b26e1b
I20260806 15:02:26.602033 7737 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a9c5995cd6a4c61b4f0fdb4844208a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a9c5995cd6a4c61b4f0fdb4844208a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44939 } }
I20260806 15:02:26.602195 7737 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a9c5995cd6a4c61b4f0fdb4844208a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:26.602232 7737 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a9c5995cd6a4c61b4f0fdb4844208a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a9c5995cd6a4c61b4f0fdb4844208a7, State: Initialized, Role: FOLLOWER
I20260806 15:02:26.602438 7737 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a9c5995cd6a4c61b4f0fdb4844208a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a9c5995cd6a4c61b4f0fdb4844208a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44939 } }
I20260806 15:02:26.602536 7737 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a9c5995cd6a4c61b4f0fdb4844208a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:02:26.602631 7737 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a9c5995cd6a4c61b4f0fdb4844208a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:02:26.602705 7737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a9c5995cd6a4c61b4f0fdb4844208a7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:26.603246 7737 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a9c5995cd6a4c61b4f0fdb4844208a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a9c5995cd6a4c61b4f0fdb4844208a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44939 } }
I20260806 15:02:26.603335 7737 leader_election.cc:302] T 00000000000000000000000000000000 P 7a9c5995cd6a4c61b4f0fdb4844208a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a9c5995cd6a4c61b4f0fdb4844208a7; no voters:
I20260806 15:02:26.603513 7737 leader_election.cc:288] T 00000000000000000000000000000000 P 7a9c5995cd6a4c61b4f0fdb4844208a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:02:26.603547 7746 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a9c5995cd6a4c61b4f0fdb4844208a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:26.603677 7746 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a9c5995cd6a4c61b4f0fdb4844208a7 [term 1 LEADER]: Becoming Leader. State: Replica: 7a9c5995cd6a4c61b4f0fdb4844208a7, State: Running, Role: LEADER
I20260806 15:02:26.603792 7737 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a9c5995cd6a4c61b4f0fdb4844208a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:26.603811 7746 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a9c5995cd6a4c61b4f0fdb4844208a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a9c5995cd6a4c61b4f0fdb4844208a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44939 } }
I20260806 15:02:26.604152 7746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a9c5995cd6a4c61b4f0fdb4844208a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a9c5995cd6a4c61b4f0fdb4844208a7. Latest consensus state: current_term: 1 leader_uuid: "7a9c5995cd6a4c61b4f0fdb4844208a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a9c5995cd6a4c61b4f0fdb4844208a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44939 } } }
I20260806 15:02:26.604238 7746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a9c5995cd6a4c61b4f0fdb4844208a7 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:26.604367 7747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a9c5995cd6a4c61b4f0fdb4844208a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a9c5995cd6a4c61b4f0fdb4844208a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a9c5995cd6a4c61b4f0fdb4844208a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44939 } } }
I20260806 15:02:26.604573 7747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a9c5995cd6a4c61b4f0fdb4844208a7 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:26.604453 7752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:02:26.605422 7752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:02:26.607508 7744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:26.601712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33023 (local address 127.0.0.1:44939)
0806 15:02:26.601914 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:26.601921 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:26.601945 (+ 24us) server_negotiation.cc:407] Connection header received
0806 15:02:26.602035 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:26.602041 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:26.602082 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:26.602205 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:26.602468 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:26.604071 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:26.605256 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:26.605842 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:26.605951 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:26.606714 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:26.606718 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:26.606719 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:26.606909 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:26.606950 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:26.606966 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:26.607039 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:26.607238 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 15:02:26.607301 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":101,"threads_started":1}
I20260806 15:02:26.607540 7752 catalog_manager.cc:1347] Generated new cluster ID: d597f9108f754744b731e38d01aaedcb
I20260806 15:02:26.607594 7752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:02:26.608476 11297 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:02:26.613235 7744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:26.609851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47823 (local address 127.0.0.1:44939)
0806 15:02:26.609885 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:26.609886 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:26.610158 (+ 272us) server_negotiation.cc:407] Connection header received
0806 15:02:26.610186 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:26.610187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:26.610198 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:26.610265 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:26.610361 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:26.611099 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:26.612026 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:26.612336 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:26.612421 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:26.612998 (+ 577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:26.612999 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:26.612999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:26.613004 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:26.613017 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:26.613022 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:26.613079 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:26.613142 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 15:02:26.613165 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 15:02:26.627568 7752 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:02:26.628207 7752 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:02:26.635232 7752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a9c5995cd6a4c61b4f0fdb4844208a7: Generated new TSK 0
I20260806 15:02:26.635428 7752 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:02:26.646311 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 7646
2026-08-06T15:02:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1495 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:02:26 dist-test-slave-68xk krb5kdc[7778](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:02:26 dist-test-slave-68xk krb5kdc[7778](info): set up 2 sockets
Aug 06 15:02:26 dist-test-slave-68xk krb5kdc[7778](info): commencing operation
krb5kdc: starting...
W20260806 15:02:27.879812 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 15:02:27 dist-test-slave-68xk krb5kdc[7778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:02:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:02:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 15:02:28.006879 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41743
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:28.091672 7794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:28.091878 7794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:28.091903 7794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:28.093075 7794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:28.093117 7794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:28.093132 7794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:28.093176 7794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:28.093204 7794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:28.094702 7794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:28.094954 7794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:28.095187 7794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 15:02:28.100322 7794 server_base.cc:1034] running on GCE node
W20260806 15:02:28.100250 7802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:28.100270 7803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:28.100286 7805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:28.100710 7794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:28.100966 7794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:28.102149 7794 hybrid_clock.cc:648] HybridClock initialized: now 1786028548102113 us; error 50 us; skew 500 ppm
Aug 06 15:02:28 dist-test-slave-68xk krb5kdc[7778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028548, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:28.104892 7794 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 15:02:28.105365 7794 webserver.cc:460] Webserver started at http://127.0.0.1:42429/ using document root <none> and password file <none>
I20260806 15:02:28.105619 7794 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:28.105695 7794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:28.105840 7794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:28.106961 7794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "f380828af5fb4e1ebdce6fcb8615d4f1"
format_stamp: "Formatted at 2026-08-06 15:02:28 on dist-test-slave-68xk"
server_key: "d118cad2357fab921d8f76c379f6af5a"
server_key_iv: "2312a219f5c3563c5789a606f56e62b5"
server_key_version: "encryptionkey@0"
I20260806 15:02:28.107328 7794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "f380828af5fb4e1ebdce6fcb8615d4f1"
format_stamp: "Formatted at 2026-08-06 15:02:28 on dist-test-slave-68xk"
server_key: "d118cad2357fab921d8f76c379f6af5a"
server_key_iv: "2312a219f5c3563c5789a606f56e62b5"
server_key_version: "encryptionkey@0"
I20260806 15:02:28.108702 7794 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:02:28.109489 7817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:28.109731 7794 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:02:28.109840 7794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "f380828af5fb4e1ebdce6fcb8615d4f1"
format_stamp: "Formatted at 2026-08-06 15:02:28 on dist-test-slave-68xk"
server_key: "d118cad2357fab921d8f76c379f6af5a"
server_key_iv: "2312a219f5c3563c5789a606f56e62b5"
server_key_version: "encryptionkey@0"
I20260806 15:02:28.109915 7794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:28.113284 7794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:28.115311 7794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:28.115468 7794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:28.119735 7794 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41743
I20260806 15:02:28.119766 7890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41743 every 8 connection(s)
I20260806 15:02:28.120105 7794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:02:28.121064 7894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:28.122169 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 7794
I20260806 15:02:28.122314 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:02:28.122574 11297 external_mini_cluster.cc:1428] Setting key fb32e0f81f5581b837a55ce953dc8570
I20260806 15:02:28.123715 7894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f380828af5fb4e1ebdce6fcb8615d4f1: Bootstrap starting.
I20260806 15:02:28.124426 7894 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f380828af5fb4e1ebdce6fcb8615d4f1: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:28.124794 7894 log.cc:824] T 00000000000000000000000000000000 P f380828af5fb4e1ebdce6fcb8615d4f1: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:28.125259 7898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:28.123124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41533 (local address 127.0.0.1:41743)
0806 15:02:28.123322 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:28.123331 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:28.123365 (+ 34us) server_negotiation.cc:407] Connection header received
0806 15:02:28.123800 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:28.123809 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:28.123845 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:28.124005 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:28.124868 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:28.124871 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:28.124872 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:28.124876 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:28.124917 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:28.124931 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:28.125010 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:28.125061 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 15:02:28.125101 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
I20260806 15:02:28.125628 7894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f380828af5fb4e1ebdce6fcb8615d4f1: No bootstrap required, opened a new log
I20260806 15:02:28.126194 11297 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:02:28.128324 7894 raft_consensus.cc:348] T 00000000000000000000000000000000 P f380828af5fb4e1ebdce6fcb8615d4f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f380828af5fb4e1ebdce6fcb8615d4f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41743 } }
I20260806 15:02:28.128490 7894 raft_consensus.cc:374] T 00000000000000000000000000000000 P f380828af5fb4e1ebdce6fcb8615d4f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:28.128517 7894 raft_consensus.cc:729] T 00000000000000000000000000000000 P f380828af5fb4e1ebdce6fcb8615d4f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f380828af5fb4e1ebdce6fcb8615d4f1, State: Initialized, Role: FOLLOWER
I20260806 15:02:28.128664 7894 consensus_queue.cc:260] T 00000000000000000000000000000000 P f380828af5fb4e1ebdce6fcb8615d4f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f380828af5fb4e1ebdce6fcb8615d4f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41743 } }
I20260806 15:02:28.128767 7894 raft_consensus.cc:388] T 00000000000000000000000000000000 P f380828af5fb4e1ebdce6fcb8615d4f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:02:28.128808 7894 raft_consensus.cc:482] T 00000000000000000000000000000000 P f380828af5fb4e1ebdce6fcb8615d4f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:02:28.128865 7894 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f380828af5fb4e1ebdce6fcb8615d4f1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:28.129082 7898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:28.127885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50085 (local address 127.0.0.1:41743)
0806 15:02:28.127924 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:28.127926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:28.128090 (+ 164us) server_negotiation.cc:407] Connection header received
0806 15:02:28.128143 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:28.128144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:28.128148 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:28.128191 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:28.128816 (+ 625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:28.128818 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:28.128818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:28.128823 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:28.128837 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:28.128841 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:28.128872 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:28.128975 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 15:02:28.128996 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 15:02:28.129513 7894 raft_consensus.cc:504] T 00000000000000000000000000000000 P f380828af5fb4e1ebdce6fcb8615d4f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f380828af5fb4e1ebdce6fcb8615d4f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41743 } }
I20260806 15:02:28.129632 7894 leader_election.cc:302] T 00000000000000000000000000000000 P f380828af5fb4e1ebdce6fcb8615d4f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f380828af5fb4e1ebdce6fcb8615d4f1; no voters:
I20260806 15:02:28.129801 7894 leader_election.cc:288] T 00000000000000000000000000000000 P f380828af5fb4e1ebdce6fcb8615d4f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:02:28.129819 7907 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f380828af5fb4e1ebdce6fcb8615d4f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:28.129969 7907 raft_consensus.cc:686] T 00000000000000000000000000000000 P f380828af5fb4e1ebdce6fcb8615d4f1 [term 1 LEADER]: Becoming Leader. State: Replica: f380828af5fb4e1ebdce6fcb8615d4f1, State: Running, Role: LEADER
I20260806 15:02:28.130093 7894 sys_catalog.cc:564] T 00000000000000000000000000000000 P f380828af5fb4e1ebdce6fcb8615d4f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:28.130087 7907 consensus_queue.cc:237] T 00000000000000000000000000000000 P f380828af5fb4e1ebdce6fcb8615d4f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f380828af5fb4e1ebdce6fcb8615d4f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41743 } }
I20260806 15:02:28.130496 7908 sys_catalog.cc:455] T 00000000000000000000000000000000 P f380828af5fb4e1ebdce6fcb8615d4f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f380828af5fb4e1ebdce6fcb8615d4f1. Latest consensus state: current_term: 1 leader_uuid: "f380828af5fb4e1ebdce6fcb8615d4f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f380828af5fb4e1ebdce6fcb8615d4f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41743 } } }
I20260806 15:02:28.130601 7908 sys_catalog.cc:458] T 00000000000000000000000000000000 P f380828af5fb4e1ebdce6fcb8615d4f1 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:28.130789 7914 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:02:28.130810 7910 sys_catalog.cc:455] T 00000000000000000000000000000000 P f380828af5fb4e1ebdce6fcb8615d4f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f380828af5fb4e1ebdce6fcb8615d4f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f380828af5fb4e1ebdce6fcb8615d4f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41743 } } }
I20260806 15:02:28.131170 7910 sys_catalog.cc:458] T 00000000000000000000000000000000 P f380828af5fb4e1ebdce6fcb8615d4f1 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:28.131315 7914 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:02:28.132893 7914 catalog_manager.cc:1347] Generated new cluster ID: d26aa3c184bb411b935889852598520e
I20260806 15:02:28.132961 7914 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:02:28.146929 7914 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:02:28.147122 7914 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:02:28.151221 7914 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f380828af5fb4e1ebdce6fcb8615d4f1: Generated new TSK 0
I20260806 15:02:28.151396 7914 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:02:28.158455 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 7794
2026-08-06T15:02:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1513 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:02:28 dist-test-slave-68xk krb5kdc[7931](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:02:28 dist-test-slave-68xk krb5kdc[7931](info): set up 2 sockets
Aug 06 15:02:28 dist-test-slave-68xk krb5kdc[7931](info): commencing operation
krb5kdc: starting...
W20260806 15:02:29.365909 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 15:02:29 dist-test-slave-68xk krb5kdc[7931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:02:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:02:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 15:02:29.487112 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41091
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:29.569835 7947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:29.570042 7947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:29.570065 7947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:29.571161 7947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:29.571197 7947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:29.571211 7947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:29.571224 7947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:29.571236 7947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:29.572609 7947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:29.572805 7947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:29.572969 7947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:29.577860 7955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:29.577867 7957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:29.577909 7954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:29.577983 7947 server_base.cc:1034] running on GCE node
I20260806 15:02:29.578298 7947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:29.578639 7947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:29.579795 7947 hybrid_clock.cc:648] HybridClock initialized: now 1786028549579775 us; error 39 us; skew 500 ppm
Aug 06 15:02:29 dist-test-slave-68xk krb5kdc[7931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028549, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:29.588145 7947 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 15:02:29.588651 7947 webserver.cc:460] Webserver started at http://127.0.0.1:37571/ using document root <none> and password file <none>
I20260806 15:02:29.588884 7947 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:29.588951 7947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:29.589098 7947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:29.590163 7947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "0a7b56818634491c990a18ed9aa3cd57"
format_stamp: "Formatted at 2026-08-06 15:02:29 on dist-test-slave-68xk"
server_key: "6e97dd2428aaed3f53f61d60bffeaeb0"
server_key_iv: "3d2b7997e0a04428f910b756d990d81a"
server_key_version: "encryptionkey@0"
I20260806 15:02:29.590564 7947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "0a7b56818634491c990a18ed9aa3cd57"
format_stamp: "Formatted at 2026-08-06 15:02:29 on dist-test-slave-68xk"
server_key: "6e97dd2428aaed3f53f61d60bffeaeb0"
server_key_iv: "3d2b7997e0a04428f910b756d990d81a"
server_key_version: "encryptionkey@0"
I20260806 15:02:29.592001 7947 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:02:29.592850 7966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:29.593179 7947 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 15:02:29.593303 7947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "0a7b56818634491c990a18ed9aa3cd57"
format_stamp: "Formatted at 2026-08-06 15:02:29 on dist-test-slave-68xk"
server_key: "6e97dd2428aaed3f53f61d60bffeaeb0"
server_key_iv: "3d2b7997e0a04428f910b756d990d81a"
server_key_version: "encryptionkey@0"
I20260806 15:02:29.593391 7947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:29.627516 7947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:29.629725 7947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:29.629886 7947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:29.634264 7947 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41091
I20260806 15:02:29.634310 8027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41091 every 8 connection(s)
I20260806 15:02:29.634696 7947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:02:29.635612 8028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:29.637892 8028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a7b56818634491c990a18ed9aa3cd57: Bootstrap starting.
I20260806 15:02:29.638473 8028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a7b56818634491c990a18ed9aa3cd57: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:29.638720 8028 log.cc:824] T 00000000000000000000000000000000 P 0a7b56818634491c990a18ed9aa3cd57: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:29.639389 8028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a7b56818634491c990a18ed9aa3cd57: No bootstrap required, opened a new log
I20260806 15:02:29.640966 8028 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a7b56818634491c990a18ed9aa3cd57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7b56818634491c990a18ed9aa3cd57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41091 } }
I20260806 15:02:29.641100 8028 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a7b56818634491c990a18ed9aa3cd57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:29.641124 8028 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a7b56818634491c990a18ed9aa3cd57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a7b56818634491c990a18ed9aa3cd57, State: Initialized, Role: FOLLOWER
I20260806 15:02:29.641224 8028 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a7b56818634491c990a18ed9aa3cd57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7b56818634491c990a18ed9aa3cd57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41091 } }
I20260806 15:02:29.641278 8028 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a7b56818634491c990a18ed9aa3cd57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:02:29.641299 8028 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a7b56818634491c990a18ed9aa3cd57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:02:29.641324 8028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a7b56818634491c990a18ed9aa3cd57 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:29.641803 8028 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a7b56818634491c990a18ed9aa3cd57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7b56818634491c990a18ed9aa3cd57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41091 } }
I20260806 15:02:29.641889 8028 leader_election.cc:302] T 00000000000000000000000000000000 P 0a7b56818634491c990a18ed9aa3cd57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a7b56818634491c990a18ed9aa3cd57; no voters:
I20260806 15:02:29.642046 8028 leader_election.cc:288] T 00000000000000000000000000000000 P 0a7b56818634491c990a18ed9aa3cd57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:02:29.642096 8031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a7b56818634491c990a18ed9aa3cd57 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:29.642246 8031 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a7b56818634491c990a18ed9aa3cd57 [term 1 LEADER]: Becoming Leader. State: Replica: 0a7b56818634491c990a18ed9aa3cd57, State: Running, Role: LEADER
I20260806 15:02:29.642292 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 7947
I20260806 15:02:29.642366 8028 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a7b56818634491c990a18ed9aa3cd57 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:29.642436 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:02:29.642385 8031 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a7b56818634491c990a18ed9aa3cd57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7b56818634491c990a18ed9aa3cd57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41091 } }
I20260806 15:02:29.642630 11297 external_mini_cluster.cc:1428] Setting key 44bdf70e0280c71579dc374a95d4849a
I20260806 15:02:29.642719 8031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a7b56818634491c990a18ed9aa3cd57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a7b56818634491c990a18ed9aa3cd57. Latest consensus state: current_term: 1 leader_uuid: "0a7b56818634491c990a18ed9aa3cd57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7b56818634491c990a18ed9aa3cd57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41091 } } }
I20260806 15:02:29.642820 8031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a7b56818634491c990a18ed9aa3cd57 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:29.643432 8032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a7b56818634491c990a18ed9aa3cd57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a7b56818634491c990a18ed9aa3cd57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7b56818634491c990a18ed9aa3cd57" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41091 } } }
I20260806 15:02:29.643532 8032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a7b56818634491c990a18ed9aa3cd57 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:29.643728 8037 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:02:29.644270 8037 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:02:29.647220 8037 catalog_manager.cc:1347] Generated new cluster ID: e6fecfe0d51b498d8321c49b3b2bf6fb
I20260806 15:02:29.647294 8037 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 15:02:29 dist-test-slave-68xk krb5kdc[7931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 15:02:29.655119 8042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:29.643139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:44681 (local address 127.0.0.1:41091)
0806 15:02:29.643901 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:29.643912 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:29.643945 (+ 33us) server_negotiation.cc:407] Connection header received
0806 15:02:29.644086 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:29.644093 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:29.644176 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:29.644319 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:29.645051 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:29.646395 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:29.647195 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:29.647832 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:29.647953 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:29.650848 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:29.650852 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:29.650853 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:29.650865 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:29.653363 (+ 2498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:29.653843 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:29.653846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:29.653846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:29.653947 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:29.654190 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:29.654191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:29.654191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:29.654506 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:29.654658 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:29.654803 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 15:02:29.654901 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":711,"thread_start_us":76,"threads_started":1}
I20260806 15:02:29.656188 11297 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:02:29.664907 8042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:29.657584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:42197 (local address 127.0.0.1:41091)
0806 15:02:29.657615 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:29.657616 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:29.657744 (+ 128us) server_negotiation.cc:407] Connection header received
0806 15:02:29.657786 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:29.657787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:29.657797 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:29.657852 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:29.658502 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:29.659235 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:29.660271 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:29.660589 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:29.660653 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:29.662085 (+ 1432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:29.662086 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:29.662086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:29.662092 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:29.663653 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:29.664088 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:29.664089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:29.664090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:29.664153 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:29.664390 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:29.664391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:29.664391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:29.664502 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:29.664596 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:29.664760 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 15:02:29.664820 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 15:02:29.684829 8037 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:02:29.685392 8037 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:02:29.692373 8037 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a7b56818634491c990a18ed9aa3cd57: Generated new TSK 0
I20260806 15:02:29.692586 8037 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:02:29.700194 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 7947
W20260806 15:02:29.709544 7929 connection.cc:535] client connection to 127.0.0.1:41091 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41091): Connection reset by peer (error 104)
2026-08-06T15:02:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1537 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:02:29 dist-test-slave-68xk krb5kdc[8064](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:02:29 dist-test-slave-68xk krb5kdc[8064](info): set up 2 sockets
Aug 06 15:02:29 dist-test-slave-68xk krb5kdc[8064](info): commencing operation
krb5kdc: starting...
W20260806 15:02:30.911707 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 15:02:31 dist-test-slave-68xk krb5kdc[8064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:02:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:02:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 15:02:31.045537 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33001
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:31.132727 8080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:31.132931 8080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:31.132953 8080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:31.134086 8080 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:31.134131 8080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:31.134146 8080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:31.134166 8080 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:31.134204 8080 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:31.135705 8080 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:31.135964 8080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:31.136240 8080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:31.141050 8086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:31.141156 8088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:31.141238 8080 server_base.cc:1034] running on GCE node
W20260806 15:02:31.141065 8085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:31.141511 8080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:31.141846 8080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:31.143033 8080 hybrid_clock.cc:648] HybridClock initialized: now 1786028551142995 us; error 47 us; skew 500 ppm
Aug 06 15:02:31 dist-test-slave-68xk krb5kdc[8064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028551, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:31.145589 8080 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 15:02:31.146021 8080 webserver.cc:460] Webserver started at http://127.0.0.1:41783/ using document root <none> and password file <none>
I20260806 15:02:31.146289 8080 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:31.146339 8080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:31.146673 8080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:31.147830 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "7aaf546696f6404e9f5e7b83ae82a684"
format_stamp: "Formatted at 2026-08-06 15:02:31 on dist-test-slave-68xk"
server_key: "d43622158f7c6a259237f5cfa0c6125d"
server_key_iv: "2cc2de943a391634dbe8544390461591"
server_key_version: "encryptionkey@0"
I20260806 15:02:31.148226 8080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "7aaf546696f6404e9f5e7b83ae82a684"
format_stamp: "Formatted at 2026-08-06 15:02:31 on dist-test-slave-68xk"
server_key: "d43622158f7c6a259237f5cfa0c6125d"
server_key_iv: "2cc2de943a391634dbe8544390461591"
server_key_version: "encryptionkey@0"
I20260806 15:02:31.149636 8080 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:02:31.150542 8097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:31.150813 8080 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:02:31.150923 8080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "7aaf546696f6404e9f5e7b83ae82a684"
format_stamp: "Formatted at 2026-08-06 15:02:31 on dist-test-slave-68xk"
server_key: "d43622158f7c6a259237f5cfa0c6125d"
server_key_iv: "2cc2de943a391634dbe8544390461591"
server_key_version: "encryptionkey@0"
I20260806 15:02:31.150995 8080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:31.173987 8080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:31.176082 8080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:31.176201 8080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:31.180269 8080 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33001
I20260806 15:02:31.180351 8163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33001 every 8 connection(s)
I20260806 15:02:31.180706 8080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:02:31.181630 8164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:31.181854 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 8080
I20260806 15:02:31.181978 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:02:31.182209 11297 external_mini_cluster.cc:1428] Setting key fe1c083fa556400fb81ddfe58aec3877
I20260806 15:02:31.185176 8164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7aaf546696f6404e9f5e7b83ae82a684: Bootstrap starting.
I20260806 15:02:31.185973 8164 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7aaf546696f6404e9f5e7b83ae82a684: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:31.186286 8164 log.cc:824] T 00000000000000000000000000000000 P 7aaf546696f6404e9f5e7b83ae82a684: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:31.187108 8164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7aaf546696f6404e9f5e7b83ae82a684: No bootstrap required, opened a new log
I20260806 15:02:31.188156 8168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:31.182820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:42261 (local address 127.0.0.1:33001)
0806 15:02:31.183026 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:31.183041 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:31.183067 (+ 26us) server_negotiation.cc:407] Connection header received
0806 15:02:31.183498 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:31.183505 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:31.183560 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:31.183674 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:31.183937 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:31.184808 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:31.185947 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:31.186548 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:31.186716 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:31.187474 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:31.187479 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:31.187479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:31.187656 (+ 177us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:31.187692 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:31.187704 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:31.187804 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:31.187898 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 15:02:31.187928 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":96,"threads_started":1}
I20260806 15:02:31.188772 8164 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7aaf546696f6404e9f5e7b83ae82a684 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aaf546696f6404e9f5e7b83ae82a684" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33001 } }
I20260806 15:02:31.188946 8164 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7aaf546696f6404e9f5e7b83ae82a684 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:31.189000 8164 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7aaf546696f6404e9f5e7b83ae82a684 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aaf546696f6404e9f5e7b83ae82a684, State: Initialized, Role: FOLLOWER
I20260806 15:02:31.189118 8164 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7aaf546696f6404e9f5e7b83ae82a684 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aaf546696f6404e9f5e7b83ae82a684" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33001 } }
I20260806 15:02:31.189221 8164 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7aaf546696f6404e9f5e7b83ae82a684 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:02:31.189254 8164 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7aaf546696f6404e9f5e7b83ae82a684 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:02:31.189309 8164 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7aaf546696f6404e9f5e7b83ae82a684 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:31.189428 11297 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:02:31.189941 8164 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7aaf546696f6404e9f5e7b83ae82a684 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aaf546696f6404e9f5e7b83ae82a684" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33001 } }
I20260806 15:02:31.190065 8164 leader_election.cc:302] T 00000000000000000000000000000000 P 7aaf546696f6404e9f5e7b83ae82a684 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7aaf546696f6404e9f5e7b83ae82a684; no voters:
I20260806 15:02:31.190265 8164 leader_election.cc:288] T 00000000000000000000000000000000 P 7aaf546696f6404e9f5e7b83ae82a684 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:02:31.190299 8171 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7aaf546696f6404e9f5e7b83ae82a684 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:31.190448 8171 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7aaf546696f6404e9f5e7b83ae82a684 [term 1 LEADER]: Becoming Leader. State: Replica: 7aaf546696f6404e9f5e7b83ae82a684, State: Running, Role: LEADER
I20260806 15:02:31.190593 8164 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7aaf546696f6404e9f5e7b83ae82a684 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:31.190620 8171 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7aaf546696f6404e9f5e7b83ae82a684 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aaf546696f6404e9f5e7b83ae82a684" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33001 } }
I20260806 15:02:31.191025 8174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7aaf546696f6404e9f5e7b83ae82a684 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7aaf546696f6404e9f5e7b83ae82a684" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aaf546696f6404e9f5e7b83ae82a684" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33001 } } }
I20260806 15:02:31.191107 8174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7aaf546696f6404e9f5e7b83ae82a684 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:31.191279 8172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7aaf546696f6404e9f5e7b83ae82a684 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7aaf546696f6404e9f5e7b83ae82a684. Latest consensus state: current_term: 1 leader_uuid: "7aaf546696f6404e9f5e7b83ae82a684" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aaf546696f6404e9f5e7b83ae82a684" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33001 } } }
I20260806 15:02:31.191344 8172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7aaf546696f6404e9f5e7b83ae82a684 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:31.191334 8181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:02:31.192014 8181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:02:31.195072 8168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:31.190985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:44565 (local address 127.0.0.1:33001)
0806 15:02:31.191027 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:31.191028 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:31.191473 (+ 445us) server_negotiation.cc:407] Connection header received
0806 15:02:31.191578 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:31.191579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:31.191592 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:31.191627 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:31.191876 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:31.192718 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:31.193629 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:31.193929 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:31.194015 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:31.194755 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:31.194756 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:31.194756 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:31.194762 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:31.194779 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:31.194784 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:31.194856 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:31.194967 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 15:02:31.194980 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 15:02:31.195710 8181 catalog_manager.cc:1347] Generated new cluster ID: b1f73fd2e79e49b081ff4b2ea17b1087
I20260806 15:02:31.195765 8181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:02:31.201828 8181 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:02:31.202585 8181 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:02:31.207616 8181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7aaf546696f6404e9f5e7b83ae82a684: Generated new TSK 0
I20260806 15:02:31.207854 8181 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:02:31.214570 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 8080
W20260806 15:02:31.224148 8062 connection.cc:535] client connection to 127.0.0.1:33001 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33001): Connection reset by peer (error 104)
2026-08-06T15:02:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1514 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:02:31 dist-test-slave-68xk krb5kdc[8200](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:02:31 dist-test-slave-68xk krb5kdc[8200](info): set up 2 sockets
Aug 06 15:02:31 dist-test-slave-68xk krb5kdc[8200](info): commencing operation
krb5kdc: starting...
W20260806 15:02:32.456316 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 15:02:32 dist-test-slave-68xk krb5kdc[8200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:02:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:02:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 15:02:32.586174 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35535
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35535
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:32.668370 8218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:32.668625 8218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:32.668668 8218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:32.669837 8218 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:32.669907 8218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:32.669931 8218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:32.669961 8218 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:32.669992 8218 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:32.671512 8218 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:32.671816 8218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:32.672056 8218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 15:02:32.677057 8218 server_base.cc:1034] running on GCE node
W20260806 15:02:32.677019 8226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:32.677008 8224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:32.677009 8229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:32.677434 8218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:32.677738 8218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:32.678905 8218 hybrid_clock.cc:648] HybridClock initialized: now 1786028552678891 us; error 31 us; skew 500 ppm
Aug 06 15:02:32 dist-test-slave-68xk krb5kdc[8200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028552, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:32.681667 8218 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 15:02:32.682107 8218 webserver.cc:460] Webserver started at http://127.0.0.1:40565/ using document root <none> and password file <none>
I20260806 15:02:32.682355 8218 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:32.682446 8218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:32.682611 8218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:32.683712 8218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "2bbc513b42914bd6844ea18e48bc4c85"
format_stamp: "Formatted at 2026-08-06 15:02:32 on dist-test-slave-68xk"
server_key: "6d1a3d01ea7868a8938e9f5d170d0891"
server_key_iv: "d128cc2b0fc4ef2e569bab32ac7a86f6"
server_key_version: "encryptionkey@0"
I20260806 15:02:32.684099 8218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "2bbc513b42914bd6844ea18e48bc4c85"
format_stamp: "Formatted at 2026-08-06 15:02:32 on dist-test-slave-68xk"
server_key: "6d1a3d01ea7868a8938e9f5d170d0891"
server_key_iv: "d128cc2b0fc4ef2e569bab32ac7a86f6"
server_key_version: "encryptionkey@0"
I20260806 15:02:32.685497 8218 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:02:32.686249 8236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:32.686488 8218 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:02:32.686596 8218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "2bbc513b42914bd6844ea18e48bc4c85"
format_stamp: "Formatted at 2026-08-06 15:02:32 on dist-test-slave-68xk"
server_key: "6d1a3d01ea7868a8938e9f5d170d0891"
server_key_iv: "d128cc2b0fc4ef2e569bab32ac7a86f6"
server_key_version: "encryptionkey@0"
I20260806 15:02:32.686681 8218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:32.690053 8218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:32.692530 8218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:32.692704 8218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:32.696789 8218 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35535
I20260806 15:02:32.696837 8296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35535 every 8 connection(s)
I20260806 15:02:32.697218 8218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:02:32.698176 8297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:32.700593 8297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bbc513b42914bd6844ea18e48bc4c85: Bootstrap starting.
I20260806 15:02:32.701087 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 8218
I20260806 15:02:32.701252 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:02:32.701393 8297 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2bbc513b42914bd6844ea18e48bc4c85: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:32.701468 11297 external_mini_cluster.cc:1428] Setting key 4730172bc0524282b9a4b5773d2722bb
I20260806 15:02:32.701707 8297 log.cc:824] T 00000000000000000000000000000000 P 2bbc513b42914bd6844ea18e48bc4c85: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:32.702690 8297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bbc513b42914bd6844ea18e48bc4c85: No bootstrap required, opened a new log
I20260806 15:02:32.704309 8302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:32.702028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:50965 (local address 127.0.0.1:35535)
0806 15:02:32.702306 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:32.702316 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:32.702350 (+ 34us) server_negotiation.cc:407] Connection header received
0806 15:02:32.702931 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:32.702939 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:32.702969 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:32.703118 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:32.703831 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:32.703835 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:32.703835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:32.703839 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:32.703890 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:32.703900 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:32.703991 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:32.704051 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 15:02:32.704097 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":167,"threads_started":1}
I20260806 15:02:32.705140 8297 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2bbc513b42914bd6844ea18e48bc4c85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bbc513b42914bd6844ea18e48bc4c85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35535 } }
I20260806 15:02:32.705329 8297 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2bbc513b42914bd6844ea18e48bc4c85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:32.705381 8297 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2bbc513b42914bd6844ea18e48bc4c85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bbc513b42914bd6844ea18e48bc4c85, State: Initialized, Role: FOLLOWER
I20260806 15:02:32.705468 11297 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:02:32.705505 8297 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2bbc513b42914bd6844ea18e48bc4c85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bbc513b42914bd6844ea18e48bc4c85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35535 } }
I20260806 15:02:32.705595 8297 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2bbc513b42914bd6844ea18e48bc4c85 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:02:32.705634 8297 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2bbc513b42914bd6844ea18e48bc4c85 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:02:32.705684 8297 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2bbc513b42914bd6844ea18e48bc4c85 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:32.706239 8297 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2bbc513b42914bd6844ea18e48bc4c85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bbc513b42914bd6844ea18e48bc4c85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35535 } }
I20260806 15:02:32.706329 8297 leader_election.cc:302] T 00000000000000000000000000000000 P 2bbc513b42914bd6844ea18e48bc4c85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2bbc513b42914bd6844ea18e48bc4c85; no voters:
I20260806 15:02:32.706550 8297 leader_election.cc:288] T 00000000000000000000000000000000 P 2bbc513b42914bd6844ea18e48bc4c85 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:02:32.706596 8304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2bbc513b42914bd6844ea18e48bc4c85 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:32.706717 8304 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2bbc513b42914bd6844ea18e48bc4c85 [term 1 LEADER]: Becoming Leader. State: Replica: 2bbc513b42914bd6844ea18e48bc4c85, State: Running, Role: LEADER
I20260806 15:02:32.706815 8297 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2bbc513b42914bd6844ea18e48bc4c85 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:32.706835 8304 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2bbc513b42914bd6844ea18e48bc4c85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bbc513b42914bd6844ea18e48bc4c85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35535 } }
I20260806 15:02:32.707187 8304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bbc513b42914bd6844ea18e48bc4c85 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2bbc513b42914bd6844ea18e48bc4c85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bbc513b42914bd6844ea18e48bc4c85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35535 } } }
I20260806 15:02:32.707336 8304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bbc513b42914bd6844ea18e48bc4c85 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:32.707614 8319 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:02:32.707538 8304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bbc513b42914bd6844ea18e48bc4c85 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bbc513b42914bd6844ea18e48bc4c85. Latest consensus state: current_term: 1 leader_uuid: "2bbc513b42914bd6844ea18e48bc4c85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bbc513b42914bd6844ea18e48bc4c85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35535 } } }
I20260806 15:02:32.707998 8304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bbc513b42914bd6844ea18e48bc4c85 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:32.708334 8302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:32.706985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:51433 (local address 127.0.0.1:35535)
0806 15:02:32.707023 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:32.707025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:32.707278 (+ 253us) server_negotiation.cc:407] Connection header received
0806 15:02:32.707344 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:32.707345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:32.707350 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:32.707407 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:32.708045 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:32.708047 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:32.708047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:32.708051 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:32.708065 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:32.708069 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:32.708100 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:32.708220 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 15:02:32.708236 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 15:02:32.708248 8319 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:02:32.710543 8319 catalog_manager.cc:1347] Generated new cluster ID: f88b2ee6034a44ada666517c69021b53
I20260806 15:02:32.710623 8319 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:02:32.714473 8319 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:02:32.714628 8319 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:02:32.730607 8319 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2bbc513b42914bd6844ea18e48bc4c85: Generated new TSK 0
I20260806 15:02:32.730805 8319 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:02:32.739261 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 8218
2026-08-06T15:02:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1525 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:02:32 dist-test-slave-68xk krb5kdc[8337](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:02:32 dist-test-slave-68xk krb5kdc[8337](info): set up 2 sockets
Aug 06 15:02:32 dist-test-slave-68xk krb5kdc[8337](info): commencing operation
krb5kdc: starting...
W20260806 15:02:33.958745 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 15:02:34 dist-test-slave-68xk krb5kdc[8337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:02:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:02:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 15:02:34.086165 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39867
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:34.169064 8353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:34.169303 8353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:34.169343 8353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:34.170580 8353 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:34.170646 8353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:34.170671 8353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:34.170701 8353 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:34.170730 8353 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:34.172312 8353 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39867
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:34.172595 8353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:34.172833 8353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:34.177922 8365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:34.177935 8367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:34.177966 8363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:34.178292 8353 server_base.cc:1034] running on GCE node
I20260806 15:02:34.178574 8353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:34.178870 8353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:34.180043 8353 hybrid_clock.cc:648] HybridClock initialized: now 1786028554180005 us; error 59 us; skew 500 ppm
Aug 06 15:02:34 dist-test-slave-68xk krb5kdc[8337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028554, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:34.182617 8353 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 15:02:34.183090 8353 webserver.cc:460] Webserver started at http://127.0.0.1:37291/ using document root <none> and password file <none>
I20260806 15:02:34.183331 8353 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:34.183403 8353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:34.183559 8353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:34.184648 8353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "dc8d6f0f23204c8fab5d600b6a34483c"
format_stamp: "Formatted at 2026-08-06 15:02:34 on dist-test-slave-68xk"
server_key: "63df54cb1c5557888968877d8a745f4b"
server_key_iv: "a1665343fa7d5b0b850315d1aa5fd9a8"
server_key_version: "encryptionkey@0"
I20260806 15:02:34.185038 8353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "dc8d6f0f23204c8fab5d600b6a34483c"
format_stamp: "Formatted at 2026-08-06 15:02:34 on dist-test-slave-68xk"
server_key: "63df54cb1c5557888968877d8a745f4b"
server_key_iv: "a1665343fa7d5b0b850315d1aa5fd9a8"
server_key_version: "encryptionkey@0"
I20260806 15:02:34.186520 8353 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:02:34.187394 8376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:34.187610 8353 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:02:34.187723 8353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "dc8d6f0f23204c8fab5d600b6a34483c"
format_stamp: "Formatted at 2026-08-06 15:02:34 on dist-test-slave-68xk"
server_key: "63df54cb1c5557888968877d8a745f4b"
server_key_iv: "a1665343fa7d5b0b850315d1aa5fd9a8"
server_key_version: "encryptionkey@0"
I20260806 15:02:34.187799 8353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:34.205761 8353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:34.207858 8353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:34.208050 8353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:34.212278 8353 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39867
I20260806 15:02:34.212366 8444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39867 every 8 connection(s)
I20260806 15:02:34.212738 8353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:02:34.213724 8445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:34.216360 8445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc8d6f0f23204c8fab5d600b6a34483c: Bootstrap starting.
I20260806 15:02:34.217006 8445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc8d6f0f23204c8fab5d600b6a34483c: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:34.217339 8445 log.cc:824] T 00000000000000000000000000000000 P dc8d6f0f23204c8fab5d600b6a34483c: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:34.218103 8445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc8d6f0f23204c8fab5d600b6a34483c: No bootstrap required, opened a new log
I20260806 15:02:34.219930 8445 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc8d6f0f23204c8fab5d600b6a34483c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc8d6f0f23204c8fab5d600b6a34483c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39867 } }
I20260806 15:02:34.220077 8445 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc8d6f0f23204c8fab5d600b6a34483c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:34.220103 8445 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc8d6f0f23204c8fab5d600b6a34483c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc8d6f0f23204c8fab5d600b6a34483c, State: Initialized, Role: FOLLOWER
I20260806 15:02:34.220306 8445 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc8d6f0f23204c8fab5d600b6a34483c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc8d6f0f23204c8fab5d600b6a34483c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39867 } }
I20260806 15:02:34.220402 8445 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc8d6f0f23204c8fab5d600b6a34483c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:02:34.220458 8445 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc8d6f0f23204c8fab5d600b6a34483c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:02:34.220510 8445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc8d6f0f23204c8fab5d600b6a34483c [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:34.221086 8445 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc8d6f0f23204c8fab5d600b6a34483c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc8d6f0f23204c8fab5d600b6a34483c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39867 } }
I20260806 15:02:34.221107 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 8353
I20260806 15:02:34.221185 8445 leader_election.cc:302] T 00000000000000000000000000000000 P dc8d6f0f23204c8fab5d600b6a34483c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc8d6f0f23204c8fab5d600b6a34483c; no voters:
I20260806 15:02:34.221232 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:02:34.221406 8445 leader_election.cc:288] T 00000000000000000000000000000000 P dc8d6f0f23204c8fab5d600b6a34483c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:02:34.221434 8451 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc8d6f0f23204c8fab5d600b6a34483c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:34.221467 11297 external_mini_cluster.cc:1428] Setting key 49f57ee1367f7da2a342ad57a05e7561
I20260806 15:02:34.221566 8451 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc8d6f0f23204c8fab5d600b6a34483c [term 1 LEADER]: Becoming Leader. State: Replica: dc8d6f0f23204c8fab5d600b6a34483c, State: Running, Role: LEADER
I20260806 15:02:34.221735 8451 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc8d6f0f23204c8fab5d600b6a34483c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc8d6f0f23204c8fab5d600b6a34483c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39867 } }
I20260806 15:02:34.221719 8445 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc8d6f0f23204c8fab5d600b6a34483c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:34.222287 8452 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc8d6f0f23204c8fab5d600b6a34483c [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc8d6f0f23204c8fab5d600b6a34483c. Latest consensus state: current_term: 1 leader_uuid: "dc8d6f0f23204c8fab5d600b6a34483c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc8d6f0f23204c8fab5d600b6a34483c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39867 } } }
I20260806 15:02:34.222375 8452 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc8d6f0f23204c8fab5d600b6a34483c [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:34.222373 8453 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc8d6f0f23204c8fab5d600b6a34483c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc8d6f0f23204c8fab5d600b6a34483c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc8d6f0f23204c8fab5d600b6a34483c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39867 } } }
I20260806 15:02:34.222501 8453 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc8d6f0f23204c8fab5d600b6a34483c [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:34.223317 8458 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:02:34.223870 8458 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:02:34.226452 8458 catalog_manager.cc:1347] Generated new cluster ID: fb8b003e12c349a4992eaa89d38ec589
I20260806 15:02:34.226507 8458 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 15:02:34 dist-test-slave-68xk krb5kdc[8337](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 15:02:34.233788 8455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:34.221992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:37735 (local address 127.0.0.1:39867)
0806 15:02:34.222193 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:34.222204 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:34.222243 (+ 39us) server_negotiation.cc:407] Connection header received
0806 15:02:34.222365 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:34.222374 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:34.222459 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:34.222618 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:34.223581 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:34.224934 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:34.225760 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:34.226354 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:34.226478 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:34.229731 (+ 3253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:34.229737 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:34.229737 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:34.229749 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:34.231826 (+ 2077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:34.232332 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:34.232335 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:34.232336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:34.232455 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:34.232738 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:34.232739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:34.232740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:34.233029 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:34.233190 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:34.233343 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 15:02:34.233562 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":94,"threads_started":1}
I20260806 15:02:34.234830 11297 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:02:34.242668 8458 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:02:34.243170 8458 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:02:34.244187 8455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:34.236237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:50003 (local address 127.0.0.1:39867)
0806 15:02:34.236278 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:34.236280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:34.236421 (+ 141us) server_negotiation.cc:407] Connection header received
0806 15:02:34.236452 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:34.236452 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:34.236463 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:34.236537 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:34.237117 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:34.238117 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:34.239120 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:34.239493 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:34.239573 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:34.240925 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:34.240926 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:34.240927 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:34.240935 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:34.242733 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:34.243224 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:34.243225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:34.243226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:34.243284 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:34.243533 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:34.243534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:34.243534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:34.243674 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:34.243756 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:34.244019 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 15:02:34.244091 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 15:02:34.254916 8458 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc8d6f0f23204c8fab5d600b6a34483c: Generated new TSK 0
I20260806 15:02:34.255134 8458 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:02:34.259122 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 8353
W20260806 15:02:34.269924 8333 connection.cc:535] client connection to 127.0.0.1:39867 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39867): Connection reset by peer (error 104)
2026-08-06T15:02:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1520 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:02:34 dist-test-slave-68xk krb5kdc[8487](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:02:34 dist-test-slave-68xk krb5kdc[8487](info): set up 2 sockets
Aug 06 15:02:34 dist-test-slave-68xk krb5kdc[8487](info): commencing operation
krb5kdc: starting...
W20260806 15:02:35.486508 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 15:02:35 dist-test-slave-68xk krb5kdc[8487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:02:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:02:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 15:02:35.613758 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45105
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:35.702405 8505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:35.702642 8505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:35.702665 8505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:35.703740 8505 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:35.703776 8505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:35.703791 8505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:35.703804 8505 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:35.703816 8505 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:35.705410 8505 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45105
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:35.705618 8505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:35.705790 8505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:35.710723 8513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:35.710719 8514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:35.710857 8518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:35.711193 8505 server_base.cc:1034] running on GCE node
I20260806 15:02:35.711486 8505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:35.711849 8505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:35.713091 8505 hybrid_clock.cc:648] HybridClock initialized: now 1786028555713089 us; error 46 us; skew 500 ppm
Aug 06 15:02:35 dist-test-slave-68xk krb5kdc[8487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028555, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:35.716006 8505 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 15:02:35.716454 8505 webserver.cc:460] Webserver started at http://127.0.0.1:33731/ using document root <none> and password file <none>
I20260806 15:02:35.716741 8505 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:35.716792 8505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:35.716960 8505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:35.718142 8505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "0bf6da2b21f041969d545f568f9821d7"
format_stamp: "Formatted at 2026-08-06 15:02:35 on dist-test-slave-68xk"
server_key: "d82b52daa379d3781a535361fbf1ace5"
server_key_iv: "2ac38b1e7f49062ad82b1b9a43300a4f"
server_key_version: "encryptionkey@0"
I20260806 15:02:35.718588 8505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "0bf6da2b21f041969d545f568f9821d7"
format_stamp: "Formatted at 2026-08-06 15:02:35 on dist-test-slave-68xk"
server_key: "d82b52daa379d3781a535361fbf1ace5"
server_key_iv: "2ac38b1e7f49062ad82b1b9a43300a4f"
server_key_version: "encryptionkey@0"
I20260806 15:02:35.720167 8505 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:02:35.721028 8527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:35.721262 8505 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:02:35.721364 8505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "0bf6da2b21f041969d545f568f9821d7"
format_stamp: "Formatted at 2026-08-06 15:02:35 on dist-test-slave-68xk"
server_key: "d82b52daa379d3781a535361fbf1ace5"
server_key_iv: "2ac38b1e7f49062ad82b1b9a43300a4f"
server_key_version: "encryptionkey@0"
I20260806 15:02:35.721426 8505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:35.744791 8505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:35.747107 8505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:35.747234 8505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:35.751839 8505 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45105
I20260806 15:02:35.751899 8601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45105 every 8 connection(s)
I20260806 15:02:35.752295 8505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:02:35.753310 8603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:35.755967 8603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bf6da2b21f041969d545f568f9821d7: Bootstrap starting.
I20260806 15:02:35.756633 8603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bf6da2b21f041969d545f568f9821d7: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:35.756954 8603 log.cc:824] T 00000000000000000000000000000000 P 0bf6da2b21f041969d545f568f9821d7: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:35.757733 8603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bf6da2b21f041969d545f568f9821d7: No bootstrap required, opened a new log
I20260806 15:02:35.759281 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 8505
I20260806 15:02:35.759387 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:02:35.759641 11297 external_mini_cluster.cc:1428] Setting key f20178f08953f9523079794bd1db86cf
I20260806 15:02:35.759796 8603 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bf6da2b21f041969d545f568f9821d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf6da2b21f041969d545f568f9821d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45105 } }
I20260806 15:02:35.760021 8603 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bf6da2b21f041969d545f568f9821d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:35.760145 8603 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bf6da2b21f041969d545f568f9821d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bf6da2b21f041969d545f568f9821d7, State: Initialized, Role: FOLLOWER
I20260806 15:02:35.760332 8603 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bf6da2b21f041969d545f568f9821d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf6da2b21f041969d545f568f9821d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45105 } }
I20260806 15:02:35.760458 8603 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bf6da2b21f041969d545f568f9821d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:02:35.760514 8603 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bf6da2b21f041969d545f568f9821d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:02:35.760561 8603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bf6da2b21f041969d545f568f9821d7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:35.761857 8603 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bf6da2b21f041969d545f568f9821d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf6da2b21f041969d545f568f9821d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45105 } }
I20260806 15:02:35.761977 8603 leader_election.cc:302] T 00000000000000000000000000000000 P 0bf6da2b21f041969d545f568f9821d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bf6da2b21f041969d545f568f9821d7; no voters:
I20260806 15:02:35.762164 8603 leader_election.cc:288] T 00000000000000000000000000000000 P 0bf6da2b21f041969d545f568f9821d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:02:35.762267 8608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bf6da2b21f041969d545f568f9821d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:35.762442 8603 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bf6da2b21f041969d545f568f9821d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:35.762499 8608 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bf6da2b21f041969d545f568f9821d7 [term 1 LEADER]: Becoming Leader. State: Replica: 0bf6da2b21f041969d545f568f9821d7, State: Running, Role: LEADER
I20260806 15:02:35.762629 8608 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bf6da2b21f041969d545f568f9821d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf6da2b21f041969d545f568f9821d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45105 } }
I20260806 15:02:35.763016 8610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bf6da2b21f041969d545f568f9821d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bf6da2b21f041969d545f568f9821d7. Latest consensus state: current_term: 1 leader_uuid: "0bf6da2b21f041969d545f568f9821d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf6da2b21f041969d545f568f9821d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45105 } } }
I20260806 15:02:35.763222 8610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bf6da2b21f041969d545f568f9821d7 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:35.763074 8609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bf6da2b21f041969d545f568f9821d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bf6da2b21f041969d545f568f9821d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bf6da2b21f041969d545f568f9821d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45105 } } }
I20260806 15:02:35.763515 8609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bf6da2b21f041969d545f568f9821d7 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:35.763548 8617 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:02:35.764190 8617 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:02:35.766592 8617 catalog_manager.cc:1347] Generated new cluster ID: 51f9afab0e4d4d00a2e3871ca923a395
I20260806 15:02:35.766718 8617 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:02:35.766991 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:35.760253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:41287 (local address 127.0.0.1:45105)
0806 15:02:35.760482 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:35.760492 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:35.760533 (+ 41us) server_negotiation.cc:407] Connection header received
0806 15:02:35.760668 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:35.760676 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:35.760742 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:35.760906 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:35.761160 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:35.762804 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:35.764129 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:35.764808 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:35.764926 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:35.766141 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:35.766145 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:35.766146 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:35.766368 (+ 222us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:35.766437 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:35.766457 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:35.766563 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:35.766718 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 15:02:35.766753 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":111,"threads_started":1}
I20260806 15:02:35.768133 11297 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:02:35.778390 8607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:35.770822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:57039 (local address 127.0.0.1:45105)
0806 15:02:35.770861 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:35.770863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:35.774612 (+ 3749us) server_negotiation.cc:407] Connection header received
0806 15:02:35.774652 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:35.774653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:35.774666 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:35.774746 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:35.774890 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:35.775742 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:35.776836 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:35.777204 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:35.777283 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:35.778101 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:35.778102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:35.778103 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:35.778108 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:35.778125 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:35.778131 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:35.778202 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:35.778284 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 15:02:35.778298 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 15:02:35.785910 8617 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:02:35.786774 8617 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:02:35.791967 8617 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bf6da2b21f041969d545f568f9821d7: Generated new TSK 0
I20260806 15:02:35.792160 8617 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:02:35.795305 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 8505
W20260806 15:02:35.817817 8485 connection.cc:535] client connection to 127.0.0.1:45105 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45105): Connection reset by peer (error 104)
2026-08-06T15:02:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1547 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:02:35 dist-test-slave-68xk krb5kdc[8637](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:02:35 dist-test-slave-68xk krb5kdc[8637](info): set up 2 sockets
Aug 06 15:02:35 dist-test-slave-68xk krb5kdc[8637](info): commencing operation
krb5kdc: starting...
W20260806 15:02:37.031199 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 15:02:37 dist-test-slave-68xk krb5kdc[8637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:02:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:02:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 15:02:37.159019 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45015
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:37.241223 8659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:37.241415 8659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:37.241438 8659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:37.242512 8659 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:37.242552 8659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:37.242566 8659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:37.242579 8659 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:37.242592 8659 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:37.244153 8659 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45015
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028287744255-11297-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:37.244397 8659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:37.244645 8659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:37.249581 8668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:37.249583 8666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:37.249697 8659 server_base.cc:1034] running on GCE node
W20260806 15:02:37.249643 8665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:37.250130 8659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:37.250465 8659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:37.251636 8659 hybrid_clock.cc:648] HybridClock initialized: now 1786028557251611 us; error 52 us; skew 500 ppm
Aug 06 15:02:37 dist-test-slave-68xk krb5kdc[8637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028557, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:37.254310 8659 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 15:02:37.254809 8659 webserver.cc:460] Webserver started at http://127.0.0.1:34503/ using document root <none> and password file <none>
I20260806 15:02:37.254994 8659 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:37.255033 8659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:37.255132 8659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:37.256162 8659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "3f36cc0c089841be971fb7021505de19"
format_stamp: "Formatted at 2026-08-06 15:02:37 on dist-test-slave-68xk"
server_key: "5940df3582880a219545844cb2219a9d"
server_key_iv: "ba5774db52127ccdb07bfc4ef9ed59a6"
server_key_version: "encryptionkey@0"
I20260806 15:02:37.256464 8659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "3f36cc0c089841be971fb7021505de19"
format_stamp: "Formatted at 2026-08-06 15:02:37 on dist-test-slave-68xk"
server_key: "5940df3582880a219545844cb2219a9d"
server_key_iv: "ba5774db52127ccdb07bfc4ef9ed59a6"
server_key_version: "encryptionkey@0"
I20260806 15:02:37.257797 8659 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:02:37.258762 8681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:37.259161 8659 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:02:37.259294 8659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "3f36cc0c089841be971fb7021505de19"
format_stamp: "Formatted at 2026-08-06 15:02:37 on dist-test-slave-68xk"
server_key: "5940df3582880a219545844cb2219a9d"
server_key_iv: "ba5774db52127ccdb07bfc4ef9ed59a6"
server_key_version: "encryptionkey@0"
I20260806 15:02:37.259387 8659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:37.263132 8659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:37.265690 8659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:37.265872 8659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:37.270614 8659 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45015
I20260806 15:02:37.270686 8754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45015 every 8 connection(s)
I20260806 15:02:37.271034 8659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:02:37.271963 8759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:37.273840 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 8659
I20260806 15:02:37.274003 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:02:37.274257 11297 external_mini_cluster.cc:1428] Setting key 736af51fa8a2200bbf6fae66980bb0b7
I20260806 15:02:37.274698 8759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f36cc0c089841be971fb7021505de19: Bootstrap starting.
I20260806 15:02:37.275463 8759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f36cc0c089841be971fb7021505de19: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:37.275857 8759 log.cc:824] T 00000000000000000000000000000000 P 3f36cc0c089841be971fb7021505de19: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:37.276674 8759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f36cc0c089841be971fb7021505de19: No bootstrap required, opened a new log
I20260806 15:02:37.277285 8763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:37.274846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:48491 (local address 127.0.0.1:45015)
0806 15:02:37.275062 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:37.275075 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:37.275118 (+ 43us) server_negotiation.cc:407] Connection header received
0806 15:02:37.275770 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:37.275779 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:37.275818 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:37.275989 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:37.276803 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:37.276807 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:37.276808 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:37.276813 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:37.276867 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:37.276882 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:37.276984 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:37.277048 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 15:02:37.277094 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":82,"threads_started":1}
I20260806 15:02:37.278365 11297 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:02:37.279184 8759 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f36cc0c089841be971fb7021505de19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f36cc0c089841be971fb7021505de19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45015 } }
I20260806 15:02:37.279338 8759 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f36cc0c089841be971fb7021505de19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:37.279393 8759 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f36cc0c089841be971fb7021505de19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f36cc0c089841be971fb7021505de19, State: Initialized, Role: FOLLOWER
I20260806 15:02:37.279577 8759 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f36cc0c089841be971fb7021505de19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f36cc0c089841be971fb7021505de19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45015 } }
I20260806 15:02:37.279682 8759 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f36cc0c089841be971fb7021505de19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:02:37.279728 8759 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f36cc0c089841be971fb7021505de19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:02:37.279772 8759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f36cc0c089841be971fb7021505de19 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:37.280506 8759 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f36cc0c089841be971fb7021505de19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f36cc0c089841be971fb7021505de19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45015 } }
I20260806 15:02:37.280647 8759 leader_election.cc:302] T 00000000000000000000000000000000 P 3f36cc0c089841be971fb7021505de19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f36cc0c089841be971fb7021505de19; no voters:
I20260806 15:02:37.280831 8759 leader_election.cc:288] T 00000000000000000000000000000000 P 3f36cc0c089841be971fb7021505de19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:02:37.280860 8773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f36cc0c089841be971fb7021505de19 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:37.281085 8759 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f36cc0c089841be971fb7021505de19 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:37.281091 8773 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f36cc0c089841be971fb7021505de19 [term 1 LEADER]: Becoming Leader. State: Replica: 3f36cc0c089841be971fb7021505de19, State: Running, Role: LEADER
I20260806 15:02:37.281214 8763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:37.279926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:52503 (local address 127.0.0.1:45015)
0806 15:02:37.279985 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:37.279987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:37.280177 (+ 190us) server_negotiation.cc:407] Connection header received
0806 15:02:37.280198 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:37.280199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:37.280203 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:37.280245 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:37.280835 (+ 590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:37.280836 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:37.280837 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:37.280841 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:37.280856 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:37.280860 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:37.280922 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:37.281144 (+ 222us) server_negotiation.cc:299] Negotiation successful
0806 15:02:37.281159 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 15:02:37.281235 8773 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f36cc0c089841be971fb7021505de19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f36cc0c089841be971fb7021505de19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45015 } }
I20260806 15:02:37.281847 8775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f36cc0c089841be971fb7021505de19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f36cc0c089841be971fb7021505de19. Latest consensus state: current_term: 1 leader_uuid: "3f36cc0c089841be971fb7021505de19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f36cc0c089841be971fb7021505de19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45015 } } }
I20260806 15:02:37.282302 8775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f36cc0c089841be971fb7021505de19 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:37.282217 8774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f36cc0c089841be971fb7021505de19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f36cc0c089841be971fb7021505de19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f36cc0c089841be971fb7021505de19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45015 } } }
I20260806 15:02:37.282546 8774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f36cc0c089841be971fb7021505de19 [sys.catalog]: This master's current role is: LEADER
W20260806 15:02:37.283001 8796 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3f36cc0c089841be971fb7021505de19: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:02:37.283069 8796 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:02:37.283120 8795 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:02:37.283358 8795 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:02:37.284909 8795 catalog_manager.cc:1347] Generated new cluster ID: 01d3c55f9d594ead9d2e82fd2d90735d
I20260806 15:02:37.284976 8795 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:02:37.320487 8795 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:02:37.320712 8795 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:02:37.326471 8795 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f36cc0c089841be971fb7021505de19: Generated new TSK 0
I20260806 15:02:37.326694 8795 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:02:37.340195 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 8659
2026-08-06T15:02:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1538 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:02:37 dist-test-slave-68xk krb5kdc[8803](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:02:37 dist-test-slave-68xk krb5kdc[8803](info): set up 2 sockets
Aug 06 15:02:37 dist-test-slave-68xk krb5kdc[8803](info): commencing operation
krb5kdc: starting...
W20260806 15:02:38.648685 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 15:02:38 dist-test-slave-68xk krb5kdc[8803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:02:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:02:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 15:02:38.772972 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:34199
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:42749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:34199
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:38.856541 8818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:38.856832 8818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:38.856879 8818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:38.858197 8818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:38.858276 8818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:38.858304 8818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:38.858330 8818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:38.858361 8818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:38.860013 8818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:42749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:34199
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:34199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:38.860299 8818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:38.860529 8818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:38.865514 8828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:38.865507 8830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:38.865590 8818 server_base.cc:1034] running on GCE node
W20260806 15:02:38.865509 8826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:38.865973 8818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:38.866315 8818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:38.867473 8818 hybrid_clock.cc:648] HybridClock initialized: now 1786028558867465 us; error 44 us; skew 500 ppm
Aug 06 15:02:38 dist-test-slave-68xk krb5kdc[8803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028558, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:38.870088 8818 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 15:02:38.870651 8818 webserver.cc:460] Webserver started at http://127.11.8.126:37901/ using document root <none> and password file <none>
I20260806 15:02:38.870893 8818 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:38.870962 8818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:38.871111 8818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:38.872175 8818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "b010cbf9a4c647fcb22d381ee7d96199"
format_stamp: "Formatted at 2026-08-06 15:02:38 on dist-test-slave-68xk"
server_key: "79c58d1fb25cdb088af2b3e67310a6b6"
server_key_iv: "802cdbccc2403c25aaf46aa1e680d65a"
server_key_version: "encryptionkey@0"
I20260806 15:02:38.872553 8818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "b010cbf9a4c647fcb22d381ee7d96199"
format_stamp: "Formatted at 2026-08-06 15:02:38 on dist-test-slave-68xk"
server_key: "79c58d1fb25cdb088af2b3e67310a6b6"
server_key_iv: "802cdbccc2403c25aaf46aa1e680d65a"
server_key_version: "encryptionkey@0"
I20260806 15:02:38.873898 8818 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:02:38.874750 8840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:38.874923 8818 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:02:38.875033 8818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "b010cbf9a4c647fcb22d381ee7d96199"
format_stamp: "Formatted at 2026-08-06 15:02:38 on dist-test-slave-68xk"
server_key: "79c58d1fb25cdb088af2b3e67310a6b6"
server_key_iv: "802cdbccc2403c25aaf46aa1e680d65a"
server_key_version: "encryptionkey@0"
I20260806 15:02:38.875139 8818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:38.888191 8818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:38.890026 8818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:38.890189 8818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:38.894539 8818 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:34199
I20260806 15:02:38.894589 8914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:34199 every 8 connection(s)
I20260806 15:02:38.894958 8818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:02:38.895865 8916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:38.898087 8916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b010cbf9a4c647fcb22d381ee7d96199: Bootstrap starting.
I20260806 15:02:38.898353 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 8818
I20260806 15:02:38.898538 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:02:38.898766 11297 external_mini_cluster.cc:1428] Setting key 53efa7359876f122a0d899cc593a8c9c
I20260806 15:02:38.898909 8916 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b010cbf9a4c647fcb22d381ee7d96199: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:38.899264 8916 log.cc:824] T 00000000000000000000000000000000 P b010cbf9a4c647fcb22d381ee7d96199: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:38.900069 8916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b010cbf9a4c647fcb22d381ee7d96199: No bootstrap required, opened a new log
I20260806 15:02:38.902453 8916 raft_consensus.cc:348] T 00000000000000000000000000000000 P b010cbf9a4c647fcb22d381ee7d96199 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b010cbf9a4c647fcb22d381ee7d96199" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34199 } }
I20260806 15:02:38.902633 8916 raft_consensus.cc:374] T 00000000000000000000000000000000 P b010cbf9a4c647fcb22d381ee7d96199 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:38.902669 8916 raft_consensus.cc:729] T 00000000000000000000000000000000 P b010cbf9a4c647fcb22d381ee7d96199 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b010cbf9a4c647fcb22d381ee7d96199, State: Initialized, Role: FOLLOWER
I20260806 15:02:38.902781 8916 consensus_queue.cc:260] T 00000000000000000000000000000000 P b010cbf9a4c647fcb22d381ee7d96199 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b010cbf9a4c647fcb22d381ee7d96199" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34199 } }
I20260806 15:02:38.902846 8916 raft_consensus.cc:388] T 00000000000000000000000000000000 P b010cbf9a4c647fcb22d381ee7d96199 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:02:38.902892 8916 raft_consensus.cc:482] T 00000000000000000000000000000000 P b010cbf9a4c647fcb22d381ee7d96199 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:02:38.902935 8916 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b010cbf9a4c647fcb22d381ee7d96199 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:38.903515 8916 raft_consensus.cc:504] T 00000000000000000000000000000000 P b010cbf9a4c647fcb22d381ee7d96199 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b010cbf9a4c647fcb22d381ee7d96199" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34199 } }
I20260806 15:02:38.903611 8916 leader_election.cc:302] T 00000000000000000000000000000000 P b010cbf9a4c647fcb22d381ee7d96199 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b010cbf9a4c647fcb22d381ee7d96199; no voters:
I20260806 15:02:38.903836 8916 leader_election.cc:288] T 00000000000000000000000000000000 P b010cbf9a4c647fcb22d381ee7d96199 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:02:38.903857 8922 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b010cbf9a4c647fcb22d381ee7d96199 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:38.903975 8922 raft_consensus.cc:686] T 00000000000000000000000000000000 P b010cbf9a4c647fcb22d381ee7d96199 [term 1 LEADER]: Becoming Leader. State: Replica: b010cbf9a4c647fcb22d381ee7d96199, State: Running, Role: LEADER
I20260806 15:02:38.904062 8922 consensus_queue.cc:237] T 00000000000000000000000000000000 P b010cbf9a4c647fcb22d381ee7d96199 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b010cbf9a4c647fcb22d381ee7d96199" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34199 } }
I20260806 15:02:38.904120 8916 sys_catalog.cc:564] T 00000000000000000000000000000000 P b010cbf9a4c647fcb22d381ee7d96199 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:38.904386 8922 sys_catalog.cc:455] T 00000000000000000000000000000000 P b010cbf9a4c647fcb22d381ee7d96199 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b010cbf9a4c647fcb22d381ee7d96199. Latest consensus state: current_term: 1 leader_uuid: "b010cbf9a4c647fcb22d381ee7d96199" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b010cbf9a4c647fcb22d381ee7d96199" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34199 } } }
I20260806 15:02:38.904470 8922 sys_catalog.cc:458] T 00000000000000000000000000000000 P b010cbf9a4c647fcb22d381ee7d96199 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:38.904397 8923 sys_catalog.cc:455] T 00000000000000000000000000000000 P b010cbf9a4c647fcb22d381ee7d96199 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b010cbf9a4c647fcb22d381ee7d96199" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b010cbf9a4c647fcb22d381ee7d96199" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 34199 } } }
I20260806 15:02:38.904734 8928 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:02:38.905356 8928 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:02:38.905941 8923 sys_catalog.cc:458] T 00000000000000000000000000000000 P b010cbf9a4c647fcb22d381ee7d96199 [sys.catalog]: This master's current role is: LEADER
Aug 06 15:02:38 dist-test-slave-68xk krb5kdc[8803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:02:38.907279 8928 catalog_manager.cc:1347] Generated new cluster ID: 508e99f833f54a57be364cc004003a1d
I20260806 15:02:38.907328 8928 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:02:38.910183 8920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:38.899348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55557 (local address 127.11.8.126:34199)
0806 15:02:38.899522 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:38.899547 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:38.899584 (+ 37us) server_negotiation.cc:407] Connection header received
0806 15:02:38.900131 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:38.900140 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:38.900210 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:38.900324 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:38.901142 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:38.901988 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:38.903164 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:38.903618 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:38.903736 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:38.906784 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:38.906788 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:38.906788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:38.906801 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:38.908563 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:38.909048 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:38.909051 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:38.909052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:38.909147 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:38.909383 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:38.909384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:38.909384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:38.909624 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:38.909816 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:38.909904 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 15:02:38.910004 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":75,"threads_started":1}
I20260806 15:02:38.911216 11297 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:02:38.924823 8920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:38.914756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41113 (local address 127.11.8.126:34199)
0806 15:02:38.914791 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:38.914793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:38.918643 (+ 3850us) server_negotiation.cc:407] Connection header received
0806 15:02:38.918711 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:38.918712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:38.918722 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:38.918796 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:38.919345 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:38.919857 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:38.920622 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:38.920851 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:38.920929 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:38.922150 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:38.922151 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:38.922152 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:38.922158 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:38.923649 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:38.924025 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:38.924026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:38.924026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:38.924083 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:38.924313 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:38.924314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:38.924314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:38.924422 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:38.924535 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:38.924653 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 15:02:38.924716 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 15:02:38.942314 8928 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:02:38.942953 8928 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:02:38.949865 8928 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b010cbf9a4c647fcb22d381ee7d96199: Generated new TSK 0
I20260806 15:02:38.950057 8928 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:02:38.957854 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 8818
W20260806 15:02:38.969983 8800 connection.cc:535] client connection to 127.11.8.126:34199 recv error: Network error: failed to read from TLS socket (remote: 127.11.8.126:34199): Connection reset by peer (error 104)
2026-08-06T15:02:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1613 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:02:39 dist-test-slave-68xk krb5kdc[8957](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:02:39 dist-test-slave-68xk krb5kdc[8957](info): set up 2 sockets
Aug 06 15:02:39 dist-test-slave-68xk krb5kdc[8957](info): commencing operation
krb5kdc: starting...
W20260806 15:02:40.190922 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 15:02:40 dist-test-slave-68xk krb5kdc[8957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028560, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:02:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:02:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 15:02:40.317917 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:43219
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:40943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:43219
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:40.401131 8972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:40.401326 8972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:40.401350 8972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:40.402516 8972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:40.402557 8972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:40.402572 8972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:40.402586 8972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:40.402661 8972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:40.404125 8972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:40943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:43219
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:43219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:40.404371 8972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:40.404599 8972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:40.409547 8978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:40.409579 8982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:40.409665 8980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:40.409905 8972 server_base.cc:1034] running on GCE node
I20260806 15:02:40.410156 8972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:40.410399 8972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:40.411597 8972 hybrid_clock.cc:648] HybridClock initialized: now 1786028560411588 us; error 34 us; skew 500 ppm
Aug 06 15:02:40 dist-test-slave-68xk krb5kdc[8957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028560, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:40.417662 8972 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 15:02:40.418118 8972 webserver.cc:460] Webserver started at http://127.11.8.126:46077/ using document root <none> and password file <none>
I20260806 15:02:40.418397 8972 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:40.418527 8972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:40.418687 8972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:40.419821 8972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "cfe1986ce6f441af985bfbcc5cc45cd5"
format_stamp: "Formatted at 2026-08-06 15:02:40 on dist-test-slave-68xk"
server_key: "baded81dd7aaed88a76de844ef3da719"
server_key_iv: "534116384d981b6b2e8dfb7502ececa9"
server_key_version: "encryptionkey@0"
I20260806 15:02:40.420202 8972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "cfe1986ce6f441af985bfbcc5cc45cd5"
format_stamp: "Formatted at 2026-08-06 15:02:40 on dist-test-slave-68xk"
server_key: "baded81dd7aaed88a76de844ef3da719"
server_key_iv: "534116384d981b6b2e8dfb7502ececa9"
server_key_version: "encryptionkey@0"
I20260806 15:02:40.421586 8972 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:02:40.422385 8990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:40.422607 8972 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:02:40.422714 8972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "cfe1986ce6f441af985bfbcc5cc45cd5"
format_stamp: "Formatted at 2026-08-06 15:02:40 on dist-test-slave-68xk"
server_key: "baded81dd7aaed88a76de844ef3da719"
server_key_iv: "534116384d981b6b2e8dfb7502ececa9"
server_key_version: "encryptionkey@0"
I20260806 15:02:40.422803 8972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:40.443248 8972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:40.446389 8972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:40.446583 8972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:40.454643 8972 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:43219
I20260806 15:02:40.454702 9055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:43219 every 8 connection(s)
I20260806 15:02:40.455073 8972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:02:40.456005 9056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:40.458405 9056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfe1986ce6f441af985bfbcc5cc45cd5: Bootstrap starting.
I20260806 15:02:40.459084 9056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfe1986ce6f441af985bfbcc5cc45cd5: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:40.459388 9056 log.cc:824] T 00000000000000000000000000000000 P cfe1986ce6f441af985bfbcc5cc45cd5: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:40.460106 9056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfe1986ce6f441af985bfbcc5cc45cd5: No bootstrap required, opened a new log
I20260806 15:02:40.461745 9056 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfe1986ce6f441af985bfbcc5cc45cd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfe1986ce6f441af985bfbcc5cc45cd5" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 43219 } }
I20260806 15:02:40.461911 9056 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfe1986ce6f441af985bfbcc5cc45cd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:40.461975 9056 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfe1986ce6f441af985bfbcc5cc45cd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfe1986ce6f441af985bfbcc5cc45cd5, State: Initialized, Role: FOLLOWER
I20260806 15:02:40.462144 9056 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfe1986ce6f441af985bfbcc5cc45cd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfe1986ce6f441af985bfbcc5cc45cd5" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 43219 } }
I20260806 15:02:40.462231 9056 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfe1986ce6f441af985bfbcc5cc45cd5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:02:40.462277 9056 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfe1986ce6f441af985bfbcc5cc45cd5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:02:40.462327 9056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfe1986ce6f441af985bfbcc5cc45cd5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:40.463270 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 8972
I20260806 15:02:40.463330 9056 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfe1986ce6f441af985bfbcc5cc45cd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfe1986ce6f441af985bfbcc5cc45cd5" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 43219 } }
I20260806 15:02:40.463430 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:02:40.463441 9056 leader_election.cc:302] T 00000000000000000000000000000000 P cfe1986ce6f441af985bfbcc5cc45cd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfe1986ce6f441af985bfbcc5cc45cd5; no voters:
I20260806 15:02:40.463615 9056 leader_election.cc:288] T 00000000000000000000000000000000 P cfe1986ce6f441af985bfbcc5cc45cd5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:02:40.463681 11297 external_mini_cluster.cc:1428] Setting key 90f4f237fd80c7a28d47c26ec5178d33
I20260806 15:02:40.463670 9060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfe1986ce6f441af985bfbcc5cc45cd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:40.463822 9060 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfe1986ce6f441af985bfbcc5cc45cd5 [term 1 LEADER]: Becoming Leader. State: Replica: cfe1986ce6f441af985bfbcc5cc45cd5, State: Running, Role: LEADER
I20260806 15:02:40.463924 9056 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfe1986ce6f441af985bfbcc5cc45cd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:40.463958 9060 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfe1986ce6f441af985bfbcc5cc45cd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfe1986ce6f441af985bfbcc5cc45cd5" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 43219 } }
I20260806 15:02:40.464442 9061 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfe1986ce6f441af985bfbcc5cc45cd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfe1986ce6f441af985bfbcc5cc45cd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfe1986ce6f441af985bfbcc5cc45cd5" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 43219 } } }
I20260806 15:02:40.464543 9061 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfe1986ce6f441af985bfbcc5cc45cd5 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:40.464442 9063 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfe1986ce6f441af985bfbcc5cc45cd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfe1986ce6f441af985bfbcc5cc45cd5. Latest consensus state: current_term: 1 leader_uuid: "cfe1986ce6f441af985bfbcc5cc45cd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfe1986ce6f441af985bfbcc5cc45cd5" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 43219 } } }
I20260806 15:02:40.464592 9063 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfe1986ce6f441af985bfbcc5cc45cd5 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:40.465564 9075 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:02:40.466285 9075 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:02:40.467887 9075 catalog_manager.cc:1347] Generated new cluster ID: ad5d37ca96f24388b1b8d7f0d78ade90
I20260806 15:02:40.467939 9075 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:02:40.469889 9066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:40.464254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34803 (local address 127.11.8.126:43219)
0806 15:02:40.464507 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:40.464517 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:40.464553 (+ 36us) server_negotiation.cc:407] Connection header received
0806 15:02:40.464684 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:40.464693 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:40.464754 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:40.464911 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:40.465177 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:40.466637 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:40.467739 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:40.468324 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:40.468431 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:40.469188 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:40.469192 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:40.469192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:40.469399 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:40.469440 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:40.469453 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:40.469529 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:40.469656 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 15:02:40.469682 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":85,"threads_started":1}
I20260806 15:02:40.470834 11297 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:02:40.475864 9066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:40.472429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57127 (local address 127.11.8.126:43219)
0806 15:02:40.472460 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:40.472462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:40.472470 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:02:40.472536 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:40.472536 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:40.472545 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:40.472597 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:40.472737 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:40.473461 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:40.474511 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:40.474830 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:40.474890 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:40.475603 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:40.475605 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:40.475605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:40.475609 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:40.475624 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:40.475628 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:40.475682 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:40.475780 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 15:02:40.475793 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 15:02:40.498844 9075 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:02:40.499430 9075 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:02:40.502447 9075 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfe1986ce6f441af985bfbcc5cc45cd5: Generated new TSK 0
I20260806 15:02:40.502666 9075 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:02:40.507078 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 8972
W20260806 15:02:40.517452 8955 connection.cc:535] client connection to 127.11.8.126:43219 recv error: Network error: failed to read from TLS socket (remote: 127.11.8.126:43219): Connection reset by peer (error 104)
2026-08-06T15:02:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1546 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:02:40 dist-test-slave-68xk krb5kdc[9102](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:02:40 dist-test-slave-68xk krb5kdc[9102](info): set up 2 sockets
Aug 06 15:02:40 dist-test-slave-68xk krb5kdc[9102](info): commencing operation
krb5kdc: starting...
W20260806 15:02:41.724529 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 15:02:41 dist-test-slave-68xk krb5kdc[9102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:02:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:02:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 15:02:41.853495 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:43669
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:41733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:43669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:41.937832 9118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:41.938056 9118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:41.938082 9118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:41.939314 9118 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:41.939359 9118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:41.939374 9118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:41.939388 9118 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:41.939399 9118 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:41.940949 9118 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:41733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:43669
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:43669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:41.941219 9118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:41.941465 9118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:41.946460 9124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:41.946484 9127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:41.946611 9118 server_base.cc:1034] running on GCE node
W20260806 15:02:41.946519 9125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:41.946977 9118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:41.947268 9118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:41.948506 9118 hybrid_clock.cc:648] HybridClock initialized: now 1786028561948492 us; error 46 us; skew 500 ppm
Aug 06 15:02:41 dist-test-slave-68xk krb5kdc[9102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028561, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:41.951393 9118 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 15:02:41.951923 9118 webserver.cc:460] Webserver started at http://127.11.8.126:45539/ using document root <none> and password file <none>
I20260806 15:02:41.952193 9118 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:41.952241 9118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:41.952411 9118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:41.953510 9118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "81a00df1f6364a47abbe4233bf191d01"
format_stamp: "Formatted at 2026-08-06 15:02:41 on dist-test-slave-68xk"
server_key: "7bd5149c3efdeedfa7c948bc0d752ea6"
server_key_iv: "27418337c11e815082857d5025c70dc0"
server_key_version: "encryptionkey@0"
I20260806 15:02:41.953888 9118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "81a00df1f6364a47abbe4233bf191d01"
format_stamp: "Formatted at 2026-08-06 15:02:41 on dist-test-slave-68xk"
server_key: "7bd5149c3efdeedfa7c948bc0d752ea6"
server_key_iv: "27418337c11e815082857d5025c70dc0"
server_key_version: "encryptionkey@0"
I20260806 15:02:41.955484 9118 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:02:41.956354 9135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:41.956591 9118 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:02:41.956722 9118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "81a00df1f6364a47abbe4233bf191d01"
format_stamp: "Formatted at 2026-08-06 15:02:41 on dist-test-slave-68xk"
server_key: "7bd5149c3efdeedfa7c948bc0d752ea6"
server_key_iv: "27418337c11e815082857d5025c70dc0"
server_key_version: "encryptionkey@0"
I20260806 15:02:41.956810 9118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:41.960266 9118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:41.962251 9118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:41.962451 9118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:41.966547 9118 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:43669
I20260806 15:02:41.966595 9199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:43669 every 8 connection(s)
I20260806 15:02:41.966944 9118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:02:41.967959 9201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:41.969175 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 9118
I20260806 15:02:41.969316 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:02:41.969523 11297 external_mini_cluster.cc:1428] Setting key 51ff3eb614d7c4f58de36296275f048c
I20260806 15:02:41.970873 9201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81a00df1f6364a47abbe4233bf191d01: Bootstrap starting.
I20260806 15:02:41.971668 9201 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81a00df1f6364a47abbe4233bf191d01: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:41.972026 9201 log.cc:824] T 00000000000000000000000000000000 P 81a00df1f6364a47abbe4233bf191d01: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:41.972101 9205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:41.970050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43673 (local address 127.11.8.126:43669)
0806 15:02:41.970264 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:41.970272 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:41.970297 (+ 25us) server_negotiation.cc:407] Connection header received
0806 15:02:41.970766 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:41.970774 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:41.970800 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:41.970918 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:41.971620 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:41.971624 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:41.971624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:41.971627 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:41.971663 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:41.971672 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:41.971781 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:41.971836 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 15:02:41.971868 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":123,"threads_started":1}
I20260806 15:02:41.972819 9201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81a00df1f6364a47abbe4233bf191d01: No bootstrap required, opened a new log
I20260806 15:02:41.973193 11297 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:02:41.975607 9201 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81a00df1f6364a47abbe4233bf191d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81a00df1f6364a47abbe4233bf191d01" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 43669 } }
I20260806 15:02:41.975805 9201 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81a00df1f6364a47abbe4233bf191d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:41.975903 9205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:41.974832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45947 (local address 127.11.8.126:43669)
0806 15:02:41.974895 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:41.974897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:41.974982 (+ 85us) server_negotiation.cc:407] Connection header received
0806 15:02:41.975069 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:41.975070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:41.975073 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:41.975118 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:41.975705 (+ 587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:41.975706 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:41.975706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:41.975710 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:41.975722 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:41.975725 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:41.975770 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:41.975829 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 15:02:41.975840 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 15:02:41.975894 9201 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81a00df1f6364a47abbe4233bf191d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81a00df1f6364a47abbe4233bf191d01, State: Initialized, Role: FOLLOWER
I20260806 15:02:41.976102 9201 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81a00df1f6364a47abbe4233bf191d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81a00df1f6364a47abbe4233bf191d01" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 43669 } }
I20260806 15:02:41.976190 9201 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81a00df1f6364a47abbe4233bf191d01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:02:41.976213 9201 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81a00df1f6364a47abbe4233bf191d01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:02:41.976246 9201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81a00df1f6364a47abbe4233bf191d01 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:41.976923 9201 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81a00df1f6364a47abbe4233bf191d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81a00df1f6364a47abbe4233bf191d01" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 43669 } }
I20260806 15:02:41.977041 9201 leader_election.cc:302] T 00000000000000000000000000000000 P 81a00df1f6364a47abbe4233bf191d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81a00df1f6364a47abbe4233bf191d01; no voters:
I20260806 15:02:41.977222 9201 leader_election.cc:288] T 00000000000000000000000000000000 P 81a00df1f6364a47abbe4233bf191d01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:02:41.977290 9217 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81a00df1f6364a47abbe4233bf191d01 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:41.977450 9217 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81a00df1f6364a47abbe4233bf191d01 [term 1 LEADER]: Becoming Leader. State: Replica: 81a00df1f6364a47abbe4233bf191d01, State: Running, Role: LEADER
I20260806 15:02:41.977492 9201 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81a00df1f6364a47abbe4233bf191d01 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:41.977562 9217 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81a00df1f6364a47abbe4233bf191d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81a00df1f6364a47abbe4233bf191d01" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 43669 } }
I20260806 15:02:41.977922 9220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81a00df1f6364a47abbe4233bf191d01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81a00df1f6364a47abbe4233bf191d01. Latest consensus state: current_term: 1 leader_uuid: "81a00df1f6364a47abbe4233bf191d01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81a00df1f6364a47abbe4233bf191d01" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 43669 } } }
I20260806 15:02:41.978031 9220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81a00df1f6364a47abbe4233bf191d01 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:41.977919 9218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81a00df1f6364a47abbe4233bf191d01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81a00df1f6364a47abbe4233bf191d01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81a00df1f6364a47abbe4233bf191d01" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 43669 } } }
I20260806 15:02:41.978173 9218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81a00df1f6364a47abbe4233bf191d01 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:41.978554 9228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:02:41.979043 9228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:02:41.980638 9228 catalog_manager.cc:1347] Generated new cluster ID: 6fb062d64f224869885de2605ab6a542
I20260806 15:02:41.980711 9228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:02:41.994640 9228 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:02:41.994866 9228 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:02:42.005322 9228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81a00df1f6364a47abbe4233bf191d01: Generated new TSK 0
I20260806 15:02:42.005563 9228 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:02:42.016469 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 9118
2026-08-06T15:02:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1510 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:02:42 dist-test-slave-68xk krb5kdc[9241](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:02:42 dist-test-slave-68xk krb5kdc[9241](info): set up 2 sockets
Aug 06 15:02:42 dist-test-slave-68xk krb5kdc[9241](info): commencing operation
krb5kdc: starting...
W20260806 15:02:43.226186 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 15:02:43 dist-test-slave-68xk krb5kdc[9241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:02:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:02:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 15:02:43.349921 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:43239
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:36495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:43239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:43.431478 9261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:43.431741 9261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:43.431785 9261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:43.432952 9261 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:43.433029 9261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:43.433054 9261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:43.433085 9261 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:43.433116 9261 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:43.434626 9261 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:36495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:43239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:43239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:43.434913 9261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:43.435151 9261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:43.440528 9269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:43.440546 9267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:43.440587 9266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:43.440847 9261 server_base.cc:1034] running on GCE node
I20260806 15:02:43.441211 9261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:43.441486 9261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:43.442750 9261 hybrid_clock.cc:648] HybridClock initialized: now 1786028563442727 us; error 43 us; skew 500 ppm
Aug 06 15:02:43 dist-test-slave-68xk krb5kdc[9241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028563, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:43.445163 9261 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 15:02:43.445623 9261 webserver.cc:460] Webserver started at http://127.11.8.126:41109/ using document root <none> and password file <none>
I20260806 15:02:43.445860 9261 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:43.445927 9261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:43.446072 9261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:43.447165 9261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "b37f43258a20464d975ac276e5116f63"
format_stamp: "Formatted at 2026-08-06 15:02:43 on dist-test-slave-68xk"
server_key: "6701b888f9b8eabd45510c2d600fdf16"
server_key_iv: "20578ea18f54806909b95318adc3960c"
server_key_version: "encryptionkey@0"
I20260806 15:02:43.447525 9261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "b37f43258a20464d975ac276e5116f63"
format_stamp: "Formatted at 2026-08-06 15:02:43 on dist-test-slave-68xk"
server_key: "6701b888f9b8eabd45510c2d600fdf16"
server_key_iv: "20578ea18f54806909b95318adc3960c"
server_key_version: "encryptionkey@0"
I20260806 15:02:43.448872 9261 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 15:02:43.449613 9279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:43.449819 9261 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 15:02:43.449925 9261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "b37f43258a20464d975ac276e5116f63"
format_stamp: "Formatted at 2026-08-06 15:02:43 on dist-test-slave-68xk"
server_key: "6701b888f9b8eabd45510c2d600fdf16"
server_key_iv: "20578ea18f54806909b95318adc3960c"
server_key_version: "encryptionkey@0"
I20260806 15:02:43.449996 9261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:43.462936 9261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:43.464843 9261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:43.465005 9261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:43.469204 9261 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:43239
I20260806 15:02:43.469238 9346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:43239 every 8 connection(s)
I20260806 15:02:43.469609 9261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:02:43.470515 9350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:43.472826 9350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b37f43258a20464d975ac276e5116f63: Bootstrap starting.
I20260806 15:02:43.473423 9350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b37f43258a20464d975ac276e5116f63: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:43.473739 9350 log.cc:824] T 00000000000000000000000000000000 P b37f43258a20464d975ac276e5116f63: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:43.474494 9350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b37f43258a20464d975ac276e5116f63: No bootstrap required, opened a new log
I20260806 15:02:43.474642 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 9261
I20260806 15:02:43.474776 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:02:43.474982 11297 external_mini_cluster.cc:1428] Setting key 4d2b92a2d392c0976f7b26074a25f53c
I20260806 15:02:43.476589 9350 raft_consensus.cc:348] T 00000000000000000000000000000000 P b37f43258a20464d975ac276e5116f63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37f43258a20464d975ac276e5116f63" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 43239 } }
I20260806 15:02:43.476761 9350 raft_consensus.cc:374] T 00000000000000000000000000000000 P b37f43258a20464d975ac276e5116f63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:43.476804 9350 raft_consensus.cc:729] T 00000000000000000000000000000000 P b37f43258a20464d975ac276e5116f63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b37f43258a20464d975ac276e5116f63, State: Initialized, Role: FOLLOWER
I20260806 15:02:43.477005 9350 consensus_queue.cc:260] T 00000000000000000000000000000000 P b37f43258a20464d975ac276e5116f63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37f43258a20464d975ac276e5116f63" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 43239 } }
I20260806 15:02:43.477104 9350 raft_consensus.cc:388] T 00000000000000000000000000000000 P b37f43258a20464d975ac276e5116f63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:02:43.477142 9350 raft_consensus.cc:482] T 00000000000000000000000000000000 P b37f43258a20464d975ac276e5116f63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:02:43.477205 9350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b37f43258a20464d975ac276e5116f63 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:43.478232 9350 raft_consensus.cc:504] T 00000000000000000000000000000000 P b37f43258a20464d975ac276e5116f63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37f43258a20464d975ac276e5116f63" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 43239 } }
I20260806 15:02:43.478366 9350 leader_election.cc:302] T 00000000000000000000000000000000 P b37f43258a20464d975ac276e5116f63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b37f43258a20464d975ac276e5116f63; no voters:
I20260806 15:02:43.478555 9350 leader_election.cc:288] T 00000000000000000000000000000000 P b37f43258a20464d975ac276e5116f63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:02:43.478610 9357 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b37f43258a20464d975ac276e5116f63 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:43.478825 9350 sys_catalog.cc:564] T 00000000000000000000000000000000 P b37f43258a20464d975ac276e5116f63 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:43.478837 9357 raft_consensus.cc:686] T 00000000000000000000000000000000 P b37f43258a20464d975ac276e5116f63 [term 1 LEADER]: Becoming Leader. State: Replica: b37f43258a20464d975ac276e5116f63, State: Running, Role: LEADER
I20260806 15:02:43.479058 9357 consensus_queue.cc:237] T 00000000000000000000000000000000 P b37f43258a20464d975ac276e5116f63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37f43258a20464d975ac276e5116f63" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 43239 } }
I20260806 15:02:43.479377 9357 sys_catalog.cc:455] T 00000000000000000000000000000000 P b37f43258a20464d975ac276e5116f63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b37f43258a20464d975ac276e5116f63. Latest consensus state: current_term: 1 leader_uuid: "b37f43258a20464d975ac276e5116f63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37f43258a20464d975ac276e5116f63" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 43239 } } }
I20260806 15:02:43.479590 9357 sys_catalog.cc:458] T 00000000000000000000000000000000 P b37f43258a20464d975ac276e5116f63 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:43.479518 9359 sys_catalog.cc:455] T 00000000000000000000000000000000 P b37f43258a20464d975ac276e5116f63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b37f43258a20464d975ac276e5116f63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b37f43258a20464d975ac276e5116f63" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 43239 } } }
I20260806 15:02:43.480017 9359 sys_catalog.cc:458] T 00000000000000000000000000000000 P b37f43258a20464d975ac276e5116f63 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:43.480118 9371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:02:43.480715 9371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:02:43.482245 9371 catalog_manager.cc:1347] Generated new cluster ID: 8ad3b3b8148a47ed9167f697604fc48f
I20260806 15:02:43.482296 9371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 15:02:43 dist-test-slave-68xk krb5kdc[9241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:02:43.487423 9356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:43.475513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45741 (local address 127.11.8.126:43239)
0806 15:02:43.475710 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:43.475719 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:43.475755 (+ 36us) server_negotiation.cc:407] Connection header received
0806 15:02:43.476573 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:43.476579 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:43.476625 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:43.476739 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:43.477587 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:43.478589 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:43.479790 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:43.480477 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:43.480580 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:43.483536 (+ 2956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:43.483540 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:43.483541 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:43.483553 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:43.485564 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:43.485987 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:43.485989 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:43.485990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:43.486124 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:43.486347 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:43.486348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:43.486348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:43.486647 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:43.486785 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:43.486920 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 15:02:43.487052 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":108,"threads_started":1}
I20260806 15:02:43.488649 11297 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:02:43.497130 9356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:43.490041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43829 (local address 127.11.8.126:43239)
0806 15:02:43.490071 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:43.490073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:43.490208 (+ 135us) server_negotiation.cc:407] Connection header received
0806 15:02:43.490260 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:43.490261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:43.490271 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:43.490320 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:43.490977 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:43.491688 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:43.492706 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:43.493021 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:43.493084 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:43.494451 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:43.494453 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:43.494453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:43.494460 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:43.495999 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:43.496393 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:43.496394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:43.496394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:43.496446 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:43.496641 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:43.496641 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:43.496641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:43.496753 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:43.496837 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:43.496972 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 15:02:43.497043 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 15:02:43.504469 9371 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:02:43.505191 9371 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:02:43.515640 9371 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b37f43258a20464d975ac276e5116f63: Generated new TSK 0
I20260806 15:02:43.515820 9371 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:02:43.519582 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 9261
2026-08-06T15:02:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1507 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:02:43 dist-test-slave-68xk krb5kdc[9394](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:02:43 dist-test-slave-68xk krb5kdc[9394](info): set up 2 sockets
Aug 06 15:02:43 dist-test-slave-68xk krb5kdc[9394](info): commencing operation
krb5kdc: starting...
W20260806 15:02:44.739827 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 15:02:44 dist-test-slave-68xk krb5kdc[9394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:02:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:02:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 15:02:44.865501 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:32903
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:44295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:32903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:44.948908 9411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:44.949124 9411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:44.949148 9411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:44.950318 9411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:44.950361 9411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:44.950376 9411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:44.950389 9411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:44.950402 9411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:44.951894 9411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:44295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:32903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:32903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:44.952159 9411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:44.952405 9411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:44.957520 9422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:44.957500 9420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:44.957508 9419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:44.957824 9411 server_base.cc:1034] running on GCE node
I20260806 15:02:44.958086 9411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:44.958485 9411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:44.959649 9411 hybrid_clock.cc:648] HybridClock initialized: now 1786028564959626 us; error 42 us; skew 500 ppm
Aug 06 15:02:44 dist-test-slave-68xk krb5kdc[9394](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028564, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:44.962466 9411 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 15:02:44.963006 9411 webserver.cc:460] Webserver started at http://127.11.8.126:43715/ using document root <none> and password file <none>
I20260806 15:02:44.963275 9411 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:44.963346 9411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:44.963511 9411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:44.964640 9411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "e9152104154848e18b6c19f7e1e58934"
format_stamp: "Formatted at 2026-08-06 15:02:44 on dist-test-slave-68xk"
server_key: "b6da343828ea683c3b2a16f343823d1c"
server_key_iv: "65672cf28cf352e1649129667ff875f8"
server_key_version: "encryptionkey@0"
I20260806 15:02:44.965037 9411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "e9152104154848e18b6c19f7e1e58934"
format_stamp: "Formatted at 2026-08-06 15:02:44 on dist-test-slave-68xk"
server_key: "b6da343828ea683c3b2a16f343823d1c"
server_key_iv: "65672cf28cf352e1649129667ff875f8"
server_key_version: "encryptionkey@0"
I20260806 15:02:44.966533 9411 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 15:02:44.967365 9434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:44.967621 9411 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:02:44.967733 9411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "e9152104154848e18b6c19f7e1e58934"
format_stamp: "Formatted at 2026-08-06 15:02:44 on dist-test-slave-68xk"
server_key: "b6da343828ea683c3b2a16f343823d1c"
server_key_iv: "65672cf28cf352e1649129667ff875f8"
server_key_version: "encryptionkey@0"
I20260806 15:02:44.967819 9411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:45.001804 9411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:45.004002 9411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:45.004212 9411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:45.009876 9411 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:32903
I20260806 15:02:45.009953 9506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:32903 every 8 connection(s)
I20260806 15:02:45.010407 9411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:02:45.010989 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 9411
I20260806 15:02:45.011152 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:02:45.011358 11297 external_mini_cluster.cc:1428] Setting key 9cf01e1202c0421611003cd969a81736
I20260806 15:02:45.011583 9507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:45.015758 9507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9152104154848e18b6c19f7e1e58934: Bootstrap starting.
I20260806 15:02:45.016489 9507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9152104154848e18b6c19f7e1e58934: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:45.016728 9507 log.cc:824] T 00000000000000000000000000000000 P e9152104154848e18b6c19f7e1e58934: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:45.017508 9507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9152104154848e18b6c19f7e1e58934: No bootstrap required, opened a new log
I20260806 15:02:45.018262 9512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:45.012148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48315 (local address 127.11.8.126:32903)
0806 15:02:45.012380 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:45.012390 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:45.012421 (+ 31us) server_negotiation.cc:407] Connection header received
0806 15:02:45.013043 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:45.013053 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:45.013120 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:45.013292 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:45.013565 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:45.015051 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:45.016233 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:45.016704 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:45.016820 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:45.017579 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:45.017585 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:45.017586 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:45.017795 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:45.017826 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:45.017838 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:45.017911 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:45.017987 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 15:02:45.018054 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":85,"threads_started":1}
I20260806 15:02:45.019246 11297 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:02:45.019785 9507 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9152104154848e18b6c19f7e1e58934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9152104154848e18b6c19f7e1e58934" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 32903 } }
I20260806 15:02:45.019959 9507 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9152104154848e18b6c19f7e1e58934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:45.020008 9507 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9152104154848e18b6c19f7e1e58934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9152104154848e18b6c19f7e1e58934, State: Initialized, Role: FOLLOWER
I20260806 15:02:45.020133 9507 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9152104154848e18b6c19f7e1e58934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9152104154848e18b6c19f7e1e58934" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 32903 } }
I20260806 15:02:45.020234 9507 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9152104154848e18b6c19f7e1e58934 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:02:45.020280 9507 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9152104154848e18b6c19f7e1e58934 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:02:45.020329 9507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9152104154848e18b6c19f7e1e58934 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:45.021077 9507 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9152104154848e18b6c19f7e1e58934 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9152104154848e18b6c19f7e1e58934" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 32903 } }
I20260806 15:02:45.021190 9507 leader_election.cc:302] T 00000000000000000000000000000000 P e9152104154848e18b6c19f7e1e58934 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9152104154848e18b6c19f7e1e58934; no voters:
I20260806 15:02:45.021346 9507 leader_election.cc:288] T 00000000000000000000000000000000 P e9152104154848e18b6c19f7e1e58934 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:02:45.021414 9522 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9152104154848e18b6c19f7e1e58934 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:45.021562 9507 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9152104154848e18b6c19f7e1e58934 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:45.021562 9522 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9152104154848e18b6c19f7e1e58934 [term 1 LEADER]: Becoming Leader. State: Replica: e9152104154848e18b6c19f7e1e58934, State: Running, Role: LEADER
I20260806 15:02:45.021726 9522 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9152104154848e18b6c19f7e1e58934 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9152104154848e18b6c19f7e1e58934" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 32903 } }
I20260806 15:02:45.022037 9524 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9152104154848e18b6c19f7e1e58934 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9152104154848e18b6c19f7e1e58934" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9152104154848e18b6c19f7e1e58934" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 32903 } } }
I20260806 15:02:45.022111 9523 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9152104154848e18b6c19f7e1e58934 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9152104154848e18b6c19f7e1e58934. Latest consensus state: current_term: 1 leader_uuid: "e9152104154848e18b6c19f7e1e58934" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9152104154848e18b6c19f7e1e58934" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 32903 } } }
I20260806 15:02:45.022233 9523 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9152104154848e18b6c19f7e1e58934 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:45.022183 9524 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9152104154848e18b6c19f7e1e58934 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:45.022746 9531 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:02:45.023312 9531 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:02:45.024252 9512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:45.020702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53237 (local address 127.11.8.126:32903)
0806 15:02:45.020744 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:45.020746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:45.020822 (+ 76us) server_negotiation.cc:407] Connection header received
0806 15:02:45.020878 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:45.020879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:45.020889 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:45.020962 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:45.021045 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:45.021710 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:45.022843 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:45.023160 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:45.023225 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:45.023987 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:45.023988 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:45.023988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:45.023993 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:45.024007 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:45.024011 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:45.024059 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:45.024128 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 15:02:45.024149 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 15:02:45.027420 9531 catalog_manager.cc:1347] Generated new cluster ID: beb061a24d3645769394b0edb1e39a8c
I20260806 15:02:45.027464 9531 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:02:45.037884 9531 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:02:45.038403 9531 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:02:45.040417 9531 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9152104154848e18b6c19f7e1e58934: Generated new TSK 0
I20260806 15:02:45.040589 9531 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:02:45.054741 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 9411
2026-08-06T15:02:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1532 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:02:45 dist-test-slave-68xk krb5kdc[9547](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:02:45 dist-test-slave-68xk krb5kdc[9547](info): set up 2 sockets
Aug 06 15:02:45 dist-test-slave-68xk krb5kdc[9547](info): commencing operation
krb5kdc: starting...
W20260806 15:02:46.268694 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 15:02:46 dist-test-slave-68xk krb5kdc[9547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:02:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:02:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 15:02:46.394202 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:35491
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:42033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:35491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:46.475651 9565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:46.475867 9565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:46.475890 9565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:46.476985 9565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:46.477021 9565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:46.477036 9565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:46.477049 9565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:46.477061 9565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:46.478511 9565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:42033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:35491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:35491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:46.478718 9565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:46.478904 9565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:46.483700 9571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:46.483744 9573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:46.483747 9570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:46.484058 9565 server_base.cc:1034] running on GCE node
I20260806 15:02:46.484316 9565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:46.484598 9565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:46.485774 9565 hybrid_clock.cc:648] HybridClock initialized: now 1786028566485761 us; error 46 us; skew 500 ppm
Aug 06 15:02:46 dist-test-slave-68xk krb5kdc[9547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028566, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:46.493422 9565 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 15:02:46.493811 9565 webserver.cc:460] Webserver started at http://127.11.8.126:41953/ using document root <none> and password file <none>
I20260806 15:02:46.494064 9565 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:46.494112 9565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:46.494278 9565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:46.495406 9565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "c332b64e48a84338802ad951bba4a267"
format_stamp: "Formatted at 2026-08-06 15:02:46 on dist-test-slave-68xk"
server_key: "b20e7cb8c7e891b421a099a1d349fe9f"
server_key_iv: "038fd49c26da23bec1e2bbba3a37c0d0"
server_key_version: "encryptionkey@0"
I20260806 15:02:46.495780 9565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "c332b64e48a84338802ad951bba4a267"
format_stamp: "Formatted at 2026-08-06 15:02:46 on dist-test-slave-68xk"
server_key: "b20e7cb8c7e891b421a099a1d349fe9f"
server_key_iv: "038fd49c26da23bec1e2bbba3a37c0d0"
server_key_version: "encryptionkey@0"
I20260806 15:02:46.497125 9565 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:02:46.497880 9585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:46.498064 9565 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:02:46.498162 9565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "c332b64e48a84338802ad951bba4a267"
format_stamp: "Formatted at 2026-08-06 15:02:46 on dist-test-slave-68xk"
server_key: "b20e7cb8c7e891b421a099a1d349fe9f"
server_key_iv: "038fd49c26da23bec1e2bbba3a37c0d0"
server_key_version: "encryptionkey@0"
I20260806 15:02:46.498260 9565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:46.501598 9565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:46.503477 9565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:46.503643 9565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:46.507683 9565 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:35491
I20260806 15:02:46.507726 9651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:35491 every 8 connection(s)
I20260806 15:02:46.508068 9565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:02:46.508985 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 9565
I20260806 15:02:46.509110 9653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:46.509140 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:02:46.509346 11297 external_mini_cluster.cc:1428] Setting key 98245692edc2bb9e0b8ab38bf963d4b5
I20260806 15:02:46.511917 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:46.509868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32915 (local address 127.11.8.126:35491)
0806 15:02:46.510050 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:46.510057 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:46.510082 (+ 25us) server_negotiation.cc:407] Connection header received
0806 15:02:46.510657 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:46.510664 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:46.510690 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:46.510805 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:46.511460 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:46.511464 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:46.511464 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:46.511468 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:46.511503 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:46.511514 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:46.511571 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:46.511659 (+ 88us) server_negotiation.cc:299] Negotiation successful
0806 15:02:46.511720 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":92,"threads_started":1}
I20260806 15:02:46.512532 9653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c332b64e48a84338802ad951bba4a267: Bootstrap starting.
I20260806 15:02:46.512791 11297 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:02:46.513203 9653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c332b64e48a84338802ad951bba4a267: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:46.513490 9653 log.cc:824] T 00000000000000000000000000000000 P c332b64e48a84338802ad951bba4a267: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:46.514215 9653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c332b64e48a84338802ad951bba4a267: No bootstrap required, opened a new log
I20260806 15:02:46.515275 9657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:46.514273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49541 (local address 127.11.8.126:35491)
0806 15:02:46.514312 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:46.514314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:46.514470 (+ 156us) server_negotiation.cc:407] Connection header received
0806 15:02:46.514543 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:46.514543 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:46.514547 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:46.514604 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:46.515078 (+ 474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:46.515078 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:46.515079 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 15:02:46.515082 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:46.515092 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 15:02:46.515096 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:46.515134 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:46.515187 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 15:02:46.515196 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 15:02:46.516527 9653 raft_consensus.cc:348] T 00000000000000000000000000000000 P c332b64e48a84338802ad951bba4a267 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c332b64e48a84338802ad951bba4a267" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35491 } }
I20260806 15:02:46.516686 9653 raft_consensus.cc:374] T 00000000000000000000000000000000 P c332b64e48a84338802ad951bba4a267 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 15:02:46.516748 9653 raft_consensus.cc:729] T 00000000000000000000000000000000 P c332b64e48a84338802ad951bba4a267 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c332b64e48a84338802ad951bba4a267, State: Initialized, Role: FOLLOWER
I20260806 15:02:46.516872 9653 consensus_queue.cc:260] T 00000000000000000000000000000000 P c332b64e48a84338802ad951bba4a267 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c332b64e48a84338802ad951bba4a267" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35491 } }
I20260806 15:02:46.516968 9653 raft_consensus.cc:388] T 00000000000000000000000000000000 P c332b64e48a84338802ad951bba4a267 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 15:02:46.517011 9653 raft_consensus.cc:482] T 00000000000000000000000000000000 P c332b64e48a84338802ad951bba4a267 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 15:02:46.517062 9653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c332b64e48a84338802ad951bba4a267 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:46.517568 9653 raft_consensus.cc:504] T 00000000000000000000000000000000 P c332b64e48a84338802ad951bba4a267 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c332b64e48a84338802ad951bba4a267" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35491 } }
I20260806 15:02:46.517691 9653 leader_election.cc:302] T 00000000000000000000000000000000 P c332b64e48a84338802ad951bba4a267 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c332b64e48a84338802ad951bba4a267; no voters:
I20260806 15:02:46.517890 9653 leader_election.cc:288] T 00000000000000000000000000000000 P c332b64e48a84338802ad951bba4a267 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 15:02:46.517925 9669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c332b64e48a84338802ad951bba4a267 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:46.518105 9669 raft_consensus.cc:686] T 00000000000000000000000000000000 P c332b64e48a84338802ad951bba4a267 [term 1 LEADER]: Becoming Leader. State: Replica: c332b64e48a84338802ad951bba4a267, State: Running, Role: LEADER
I20260806 15:02:46.518139 9653 sys_catalog.cc:564] T 00000000000000000000000000000000 P c332b64e48a84338802ad951bba4a267 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:46.518245 9669 consensus_queue.cc:237] T 00000000000000000000000000000000 P c332b64e48a84338802ad951bba4a267 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c332b64e48a84338802ad951bba4a267" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35491 } }
I20260806 15:02:46.518682 9670 sys_catalog.cc:455] T 00000000000000000000000000000000 P c332b64e48a84338802ad951bba4a267 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c332b64e48a84338802ad951bba4a267" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c332b64e48a84338802ad951bba4a267" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35491 } } }
I20260806 15:02:46.518780 9670 sys_catalog.cc:458] T 00000000000000000000000000000000 P c332b64e48a84338802ad951bba4a267 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:46.519003 9679 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:02:46.518978 9672 sys_catalog.cc:455] T 00000000000000000000000000000000 P c332b64e48a84338802ad951bba4a267 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c332b64e48a84338802ad951bba4a267. Latest consensus state: current_term: 1 leader_uuid: "c332b64e48a84338802ad951bba4a267" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c332b64e48a84338802ad951bba4a267" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 35491 } } }
I20260806 15:02:46.519177 9672 sys_catalog.cc:458] T 00000000000000000000000000000000 P c332b64e48a84338802ad951bba4a267 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:46.519518 9679 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:02:46.521195 9679 catalog_manager.cc:1347] Generated new cluster ID: 76b1027785b6467ab3be6d5db8c270ea
I20260806 15:02:46.521247 9679 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:02:46.534138 9679 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:02:46.534325 9679 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:02:46.540977 9679 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c332b64e48a84338802ad951bba4a267: Generated new TSK 0
I20260806 15:02:46.541172 9679 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:02:46.549440 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 9565
2026-08-06T15:02:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1494 ms)
[----------] 18 tests from AuthTokenIssuingTest (27467 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:02:46 dist-test-slave-68xk krb5kdc[9699](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:02:46 dist-test-slave-68xk krb5kdc[9699](info): set up 2 sockets
Aug 06 15:02:46 dist-test-slave-68xk krb5kdc[9699](info): commencing operation
krb5kdc: starting...
W20260806 15:02:47.837368 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.261s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 15:02:47 dist-test-slave-68xk krb5kdc[9699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:02:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:02:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 15:02:47.965292 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:40203
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:46809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:40203,127.11.8.125:35777,127.11.8.124:36135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:48.048069 9716 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 15:02:48.048331 9716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:48.048362 9716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:48.048405 9716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:48.049558 9716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:48.049626 9716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:48.049651 9716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:48.049674 9716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:48.049701 9716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:48.051214 9716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:46809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:40203,127.11.8.125:35777,127.11.8.124:36135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:40203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:48.051502 9716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:48.051740 9716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:48.056653 9725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:48.056675 9722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:48.056704 9721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:48.056733 9716 server_base.cc:1034] running on GCE node
I20260806 15:02:48.057091 9716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:48.057384 9716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:48.058531 9716 hybrid_clock.cc:648] HybridClock initialized: now 1786028568058509 us; error 40 us; skew 500 ppm
Aug 06 15:02:48 dist-test-slave-68xk krb5kdc[9699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028568, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:48.061043 9716 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 15:02:48.061558 9716 webserver.cc:460] Webserver started at http://127.11.8.126:37923/ using document root <none> and password file <none>
I20260806 15:02:48.061810 9716 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:48.061879 9716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:48.062019 9716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:48.063182 9716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "b3eec0cf1151414589bc0a48cd4362b6"
format_stamp: "Formatted at 2026-08-06 15:02:48 on dist-test-slave-68xk"
server_key: "649492c0b0f569a111b6bba21938ed15"
server_key_iv: "13cd2fb97fa0086748637faa748aae42"
server_key_version: "encryptionkey@0"
I20260806 15:02:48.063560 9716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "b3eec0cf1151414589bc0a48cd4362b6"
format_stamp: "Formatted at 2026-08-06 15:02:48 on dist-test-slave-68xk"
server_key: "649492c0b0f569a111b6bba21938ed15"
server_key_iv: "13cd2fb97fa0086748637faa748aae42"
server_key_version: "encryptionkey@0"
I20260806 15:02:48.064930 9716 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 15:02:48.065676 9732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:48.065871 9716 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:02:48.065980 9716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "b3eec0cf1151414589bc0a48cd4362b6"
format_stamp: "Formatted at 2026-08-06 15:02:48 on dist-test-slave-68xk"
server_key: "649492c0b0f569a111b6bba21938ed15"
server_key_iv: "13cd2fb97fa0086748637faa748aae42"
server_key_version: "encryptionkey@0"
I20260806 15:02:48.066056 9716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:48.085526 9716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:48.087751 9716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:48.087930 9716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:48.092439 9716 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:40203
I20260806 15:02:48.092487 9808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:40203 every 8 connection(s)
I20260806 15:02:48.092872 9716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:02:48.093917 9811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:48.095593 9811 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40203 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:02:48.100595 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 9716
I20260806 15:02:48.100719 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:02:48.100942 11297 external_mini_cluster.cc:1428] Setting key 4ebeb8ea9adf438b3b9c91883312c73f
Aug 06 15:02:48 dist-test-slave-68xk krb5kdc[9699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028568, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:02:48.105968 9814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.096403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54142 (local address 127.11.8.126:40203)
0806 15:02:48.096564 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.096569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.096631 (+ 62us) server_negotiation.cc:407] Connection header received
0806 15:02:48.097130 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.097132 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.097149 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.097226 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:48.097783 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.098697 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.099533 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.099848 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.099936 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.102498 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:48.102506 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:48.102506 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:48.102512 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:48.104114 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.104600 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.104603 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.104604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.104709 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.104986 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.104987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.104987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.105264 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:48.105339 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:48.105612 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 15:02:48.105777 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":87,"threads_started":1}
I20260806 15:02:48.106081 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.096295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40203 (local address 127.0.0.1:54142)
0806 15:02:48.096496 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:48.096538 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:48.096985 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:48.097270 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:48.097273 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:48.097527 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:48.097679 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:48.097682 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.098781 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.098782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:48.099448 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:48.099449 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.099943 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.099944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:48.099998 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.100473 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:02:48.100481 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:48.102373 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:48.104192 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:48.104200 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:48.104202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:48.104490 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:48.104793 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:48.104794 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:48.104795 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:48.104912 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:48.105352 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:48.105360 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:48.105451 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 15:02:48.105562 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 15:02:48.105893 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":143,"thread_start_us":107,"threads_started":1}
I20260806 15:02:48.106961 9811 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.125" port: 35777 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:02:48.107388 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.107241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:35777 (local address 127.0.0.1:51778)
0806 15:02:48.107298 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:48.107354 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.125:35777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 15:02:48.107623 9734 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.8.125:35777: connect: Connection refused (error 111)
I20260806 15:02:48.108088 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.107978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:35777 (local address 127.0.0.1:51790)
0806 15:02:48.108028 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:48.108064 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.125:35777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 15:02:48.108274 9811 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.125:35777: Network error: Client connection negotiation failed: client connection to 127.11.8.125:35777: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.11.8.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.125.keytab.
Entry for principal kudu/127.11.8.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.125.keytab.
I20260806 15:02:48.129022 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.125:35777
--webserver_interface=127.11.8.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.125.keytab
--principal=kudu/127.11.8.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:46809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:40203,127.11.8.125:35777,127.11.8.124:36135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:02:48.157550 9811 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.125" port: 35777 } attempt: 1
I20260806 15:02:48.158128 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.157965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:35777 (local address 127.0.0.1:51798)
0806 15:02:48.158044 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:48.158092 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.125:35777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 15:02:48.158579 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.158477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:35777 (local address 127.0.0.1:51806)
0806 15:02:48.158524 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:48.158556 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.125:35777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 15:02:48.158684 9811 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.125:35777: Network error: Client connection negotiation failed: client connection to 127.11.8.125:35777: connect: Connection refused (error 111)
W20260806 15:02:48.216547 9819 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 15:02:48.216806 9819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:48.216826 9819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:48.216849 9819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:48.218183 9819 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:48.218230 9819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:48.218246 9819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:48.218294 9819 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:48.218328 9819 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:48.219995 9819 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:46809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:40203,127.11.8.125:35777,127.11.8.124:36135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.125.keytab
--principal=kudu/127.11.8.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.125:35777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.8.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:48.220273 9819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:48.220530 9819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 15:02:48.226352 9819 server_base.cc:1034] running on GCE node
W20260806 15:02:48.226291 9831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:48.226303 9827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:48.226322 9828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:48.226825 9819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:48.226826 9811 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.125" port: 35777 } attempt: 2
I20260806 15:02:48.227216 9819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:48.227252 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.227127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:35777 (local address 127.0.0.1:51812)
0806 15:02:48.227191 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:48.227223 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.125:35777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 15:02:48.227619 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.227531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:35777 (local address 127.0.0.1:51824)
0806 15:02:48.227578 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:48.227602 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.125:35777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260806 15:02:48.227771 9811 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.125:35777: Network error: Client connection negotiation failed: client connection to 127.11.8.125:35777: connect: Connection refused (error 111)
I20260806 15:02:48.228410 9819 hybrid_clock.cc:648] HybridClock initialized: now 1786028568228394 us; error 28 us; skew 500 ppm
Aug 06 15:02:48 dist-test-slave-68xk krb5kdc[9699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028568, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:48.231204 9819 init.cc:377] Logged in from keytab as kudu/127.11.8.125@KRBTEST.COM (short username kudu)
I20260806 15:02:48.231720 9819 webserver.cc:460] Webserver started at http://127.11.8.125:40085/ using document root <none> and password file <none>
I20260806 15:02:48.231964 9819 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:48.232036 9819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:48.232205 9819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:48.233354 9819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-1/data/instance:
uuid: "49436d0f114b43f1ac175c84c2f8b98e"
format_stamp: "Formatted at 2026-08-06 15:02:48 on dist-test-slave-68xk"
server_key: "ce7e0086984747f15bf523eb63a77542"
server_key_iv: "71a0bfd841c4415cf90edbd00963523a"
server_key_version: "encryptionkey@0"
I20260806 15:02:48.233724 9819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-1/wal/instance:
uuid: "49436d0f114b43f1ac175c84c2f8b98e"
format_stamp: "Formatted at 2026-08-06 15:02:48 on dist-test-slave-68xk"
server_key: "ce7e0086984747f15bf523eb63a77542"
server_key_iv: "71a0bfd841c4415cf90edbd00963523a"
server_key_version: "encryptionkey@0"
I20260806 15:02:48.235280 9819 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:02:48.236074 9840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:48.236255 9819 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 15:02:48.236366 9819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-1/wal
uuid: "49436d0f114b43f1ac175c84c2f8b98e"
format_stamp: "Formatted at 2026-08-06 15:02:48 on dist-test-slave-68xk"
server_key: "ce7e0086984747f15bf523eb63a77542"
server_key_iv: "71a0bfd841c4415cf90edbd00963523a"
server_key_version: "encryptionkey@0"
I20260806 15:02:48.236443 9819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:48.253289 9819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:48.255892 9819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:48.256065 9819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:48.260376 9819 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.125:35777
I20260806 15:02:48.260429 9907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.125:35777 every 8 connection(s)
I20260806 15:02:48.260808 9819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-1/data/info.pb
I20260806 15:02:48.261886 9908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:48.263559 9908 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40203 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:02:48.264495 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 9819
I20260806 15:02:48.264614 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-1/wal/instance
I20260806 15:02:48.264807 11297 external_mini_cluster.cc:1428] Setting key e4542aacb26d6ddb71df09c1498d5f68
Aug 06 15:02:48 dist-test-slave-68xk krb5kdc[9699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028568, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.125@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.8.124@KRBTEST.COM; defaulting to no policy
I20260806 15:02:48.274829 9814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.264318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54144 (local address 127.11.8.126:40203)
0806 15:02:48.264393 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.264396 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.264603 (+ 207us) server_negotiation.cc:407] Connection header received
0806 15:02:48.265205 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.265207 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.265224 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.265298 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:48.265903 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.266707 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.268255 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.268545 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.268611 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.271446 (+ 2835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:48.271447 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:48.271448 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:48.271457 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:48.273146 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.273666 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.273668 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.273669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.273745 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.274066 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.274068 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.274068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.274233 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:48.274318 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:48.274675 (+ 357us) server_negotiation.cc:299] Negotiation successful
0806 15:02:48.274758 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 15:02:48.275144 9910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.264275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40203 (local address 127.0.0.1:54144)
0806 15:02:48.264443 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:48.264486 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:48.265100 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:48.265371 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:48.265375 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:48.265621 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:48.265814 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:48.265817 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.266809 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.266810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:48.268138 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:48.268142 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.268656 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.268657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:48.268808 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.269344 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:02:48.269355 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:48.271341 (+ 1986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:48.273284 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:48.273294 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:48.273296 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:48.273563 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:48.273825 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:48.273826 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:48.273827 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:48.273952 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:48.274316 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:48.274323 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:48.274518 (+ 195us) client_negotiation.cc:769] Sending connection context
0806 15:02:48.274645 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 15:02:48.274788 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
I20260806 15:02:48.275813 9908 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.125" port: 35777 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.124@KRBTEST.COM" created.
Aug 06 15:02:48 dist-test-slave-68xk krb5kdc[9699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028568, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.125@KRBTEST.COM for kudu/127.11.8.125@KRBTEST.COM
I20260806 15:02:48.284178 9910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.276077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:35777 (local address 127.0.0.1:51830)
0806 15:02:48.276173 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:48.276184 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:48.276262 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:48.276534 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:48.276535 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:48.276677 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:48.276777 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:48.276779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.277916 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.277917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:48.278764 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:48.278767 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.279312 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.279314 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:48.279381 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.279873 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:02:48.279879 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:48.281208 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:48.283085 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:48.283086 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:48.283086 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:48.283261 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:48.283488 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:48.283489 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:48.283490 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:48.283550 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:48.283848 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:48.283849 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:48.283890 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 15:02:48.283953 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 15:02:48.284058 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260806 15:02:48.284274 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.276182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51830 (local address 127.11.8.125:35777)
0806 15:02:48.276361 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.276369 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.276391 (+ 22us) server_negotiation.cc:407] Connection header received
0806 15:02:48.276412 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.276414 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.276425 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.276535 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:48.276866 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.277821 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.278866 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.279192 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.279314 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.281300 (+ 1986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:48.281310 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:48.281310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:48.281320 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:48.282980 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.283357 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.283360 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.283361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.283418 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.283612 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.283613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.283614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.283766 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:48.283860 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:48.283993 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 15:02:48.284131 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":146,"thread_start_us":113,"threads_started":1}
I20260806 15:02:48.285089 9908 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36135 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:02:48.285480 9910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.285352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36135 (local address 127.0.0.1:60336)
0806 15:02:48.285403 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:48.285456 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 15:02:48.285642 9841 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36135: connect: Connection refused (error 111)
I20260806 15:02:48.286157 9910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.286056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36135 (local address 127.0.0.1:60338)
0806 15:02:48.286108 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:48.286133 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 15:02:48.286268 9908 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.124:36135: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36135: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.124.keytab.
Entry for principal kudu/127.11.8.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.124.keytab.
I20260806 15:02:48.295437 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.124:36135
--webserver_interface=127.11.8.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.124.keytab
--principal=kudu/127.11.8.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:46809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:40203,127.11.8.125:35777,127.11.8.124:36135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:02:48.318924 9811 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.125" port: 35777 } attempt: 3
Aug 06 15:02:48 dist-test-slave-68xk krb5kdc[9699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028568, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for kudu/127.11.8.125@KRBTEST.COM
I20260806 15:02:48.326609 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.319360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:35777 (local address 127.0.0.1:51846)
0806 15:02:48.319429 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:48.319442 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:48.319522 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:48.319670 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:48.319671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:48.319856 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:48.319931 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:48.319933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.320647 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.320649 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:48.321184 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:48.321186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.321591 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.321592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:48.321678 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.322204 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:02:48.322210 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:48.323604 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:48.325478 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:48.325479 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:48.325479 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:48.325678 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:48.325915 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:48.325916 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:48.325916 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:48.325987 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:48.326273 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:48.326274 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:48.326319 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 15:02:48.326379 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 15:02:48.326528 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 15:02:48.326625 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.319450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51846 (local address 127.11.8.125:35777)
0806 15:02:48.319530 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.319532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.319546 (+ 14us) server_negotiation.cc:407] Connection header received
0806 15:02:48.319581 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.319582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.319600 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.319663 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:48.320029 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.320549 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.321282 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.321511 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.321593 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.323697 (+ 2104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:48.323699 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:48.323699 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:48.323707 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:48.325369 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.325765 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.325767 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.325767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.325836 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.326060 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.326061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.326062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.326201 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:48.326273 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:48.326443 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 15:02:48.326529 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 15:02:48.327167 9811 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36135 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:02:48.327558 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.327435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36135 (local address 127.0.0.1:60346)
0806 15:02:48.327493 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:48.327529 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 15:02:48.327970 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.327867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36135 (local address 127.0.0.1:60350)
0806 15:02:48.327917 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:48.327947 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 15:02:48.328178 9811 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.124:36135: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36135: connect: Connection refused (error 111)
I20260806 15:02:48.335476 9908 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36135 } attempt: 1
I20260806 15:02:48.335862 9910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.335744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36135 (local address 127.0.0.1:60356)
0806 15:02:48.335802 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:48.335836 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 15:02:48.336222 9910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.336117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36135 (local address 127.0.0.1:60366)
0806 15:02:48.336169 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:48.336193 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 15:02:48.336308 9908 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.124:36135: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36135: connect: Connection refused (error 111)
I20260806 15:02:48.359354 9811 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36135 } attempt: 1
I20260806 15:02:48.359810 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.359672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36135 (local address 127.0.0.1:60372)
0806 15:02:48.359749 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:48.359781 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 15:02:48.360179 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.360084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36135 (local address 127.0.0.1:60384)
0806 15:02:48.360129 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:48.360154 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 15:02:48.360307 9811 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.124:36135: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36135: connect: Connection refused (error 111)
W20260806 15:02:48.382652 9917 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 15:02:48.382882 9917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:48.382913 9917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:48.382973 9917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:48.384099 9917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:48.384160 9917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:48.384184 9917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:48.384213 9917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:48.384239 9917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:48.385794 9917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:46809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:40203,127.11.8.125:35777,127.11.8.124:36135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.124.keytab
--principal=kudu/127.11.8.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.124:36135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.8.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:48.386085 9917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:48.386315 9917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:48.391244 9932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:48.391252 9929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:48.391256 9930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:48.391304 9917 server_base.cc:1034] running on GCE node
I20260806 15:02:48.391707 9917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:48.391969 9917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:48.393122 9917 hybrid_clock.cc:648] HybridClock initialized: now 1786028568393108 us; error 31 us; skew 500 ppm
Aug 06 15:02:48 dist-test-slave-68xk krb5kdc[9699](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028568, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:48.395897 9917 init.cc:377] Logged in from keytab as kudu/127.11.8.124@KRBTEST.COM (short username kudu)
I20260806 15:02:48.396356 9917 webserver.cc:460] Webserver started at http://127.11.8.124:38793/ using document root <none> and password file <none>
I20260806 15:02:48.396616 9917 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:48.396662 9917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:48.396844 9917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:48.397933 9917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-2/data/instance:
uuid: "e1c5a048f185471f99d19de7aec1c17b"
format_stamp: "Formatted at 2026-08-06 15:02:48 on dist-test-slave-68xk"
server_key: "1086fe4173eaaf22fe32d88beefa4361"
server_key_iv: "a072204c4eaa1f8b7fe08e0e80bdb4cd"
server_key_version: "encryptionkey@0"
I20260806 15:02:48.398315 9917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-2/wal/instance:
uuid: "e1c5a048f185471f99d19de7aec1c17b"
format_stamp: "Formatted at 2026-08-06 15:02:48 on dist-test-slave-68xk"
server_key: "1086fe4173eaaf22fe32d88beefa4361"
server_key_iv: "a072204c4eaa1f8b7fe08e0e80bdb4cd"
server_key_version: "encryptionkey@0"
I20260806 15:02:48.399765 9917 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:02:48.400545 9940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:48.400740 9917 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:02:48.400838 9917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-2/wal
uuid: "e1c5a048f185471f99d19de7aec1c17b"
format_stamp: "Formatted at 2026-08-06 15:02:48 on dist-test-slave-68xk"
server_key: "1086fe4173eaaf22fe32d88beefa4361"
server_key_iv: "a072204c4eaa1f8b7fe08e0e80bdb4cd"
server_key_version: "encryptionkey@0"
I20260806 15:02:48.400926 9917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:48.404449 9908 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36135 } attempt: 2
I20260806 15:02:48.404898 9910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.404767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36135 (local address 127.0.0.1:60396)
0806 15:02:48.404828 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:48.404867 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 15:02:48.405303 9910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.405217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36135 (local address 127.0.0.1:60398)
0806 15:02:48.405262 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:48.405286 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 15:02:48.405432 9908 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.124:36135: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36135: connect: Connection refused (error 111)
I20260806 15:02:48.419358 9917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:48.421320 9917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:48.421473 9917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:48.425573 9917 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.124:36135
I20260806 15:02:48.425639 10004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.124:36135 every 8 connection(s)
I20260806 15:02:48.426014 9917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-2/data/info.pb
I20260806 15:02:48.427207 10006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:48.428906 10006 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40203 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:02:48.431386 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 9917
I20260806 15:02:48.431470 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786028287744255-11297-0/minicluster-data/master-2/wal/instance
I20260806 15:02:48.431681 11297 external_mini_cluster.cc:1428] Setting key 3aacd46b59c08508d418f2a1c4d0694b
I20260806 15:02:48.435465 9811 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36135 } attempt: 2
Aug 06 15:02:48 dist-test-slave-68xk krb5kdc[9699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028568, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.124@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
Aug 06 15:02:48 dist-test-slave-68xk krb5kdc[9699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.124@KRBTEST.COM
Aug 06 15:02:48 dist-test-slave-68xk krb5kdc[9699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028568, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for kudu/127.11.8.124@KRBTEST.COM
I20260806 15:02:48.440418 9814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.429736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54152 (local address 127.11.8.126:40203)
0806 15:02:48.429801 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.429803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.429979 (+ 176us) server_negotiation.cc:407] Connection header received
0806 15:02:48.430534 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.430535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.430546 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.430616 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:48.431304 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.431904 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.433494 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.433683 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.433750 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.436391 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:48.436392 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:48.436392 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:48.436399 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:48.438058 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.438681 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.438682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.438682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.438747 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.439627 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.439628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.439629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.439778 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:48.439853 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:48.440227 (+ 374us) server_negotiation.cc:299] Negotiation successful
0806 15:02:48.440309 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 15:02:48.440547 10010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.429642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:40203 (local address 127.0.0.1:54152)
0806 15:02:48.429837 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:48.429884 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:48.430377 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:48.430688 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:48.430691 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:48.430955 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:48.431197 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:48.431204 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.432030 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.432032 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:48.433365 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:48.433367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.433793 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.433795 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:48.433925 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.434540 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:02:48.434548 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:48.436272 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:48.438141 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:48.438150 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:48.438152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:48.438557 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:48.438823 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:48.438824 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:48.438825 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:48.438938 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:48.439861 (+ 923us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:48.439868 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:48.440045 (+ 177us) client_negotiation.cc:769] Sending connection context
0806 15:02:48.440154 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 15:02:48.440324 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":113,"threads_started":1}
I20260806 15:02:48.441301 10006 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.125" port: 35777 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:02:48.441918 10012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.432127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60402 (local address 127.11.8.124:36135)
0806 15:02:48.432291 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.432297 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.432396 (+ 99us) server_negotiation.cc:407] Connection header received
0806 15:02:48.432423 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.432425 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.432449 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.432542 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:48.433188 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.434202 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.435032 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.435393 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.435452 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.438309 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:48.438318 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:48.438318 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:48.438325 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:48.440294 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.440778 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.440782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.440782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.440887 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.441108 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.441109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.441109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.441287 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:48.441450 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:48.441548 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 15:02:48.441618 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
I20260806 15:02:48.444049 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.435765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36135 (local address 127.0.0.1:60408)
0806 15:02:48.435820 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:48.435830 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:48.435870 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:48.436083 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:48.436084 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:48.436239 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:48.436272 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:48.436273 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.437133 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.437135 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:48.437963 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:48.437967 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.438470 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.438472 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:48.438537 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.439065 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:02:48.439070 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:48.440681 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:48.442949 (+ 2268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:48.442950 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:48.442951 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:48.443124 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:48.443355 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:48.443356 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:48.443357 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:48.443420 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:48.443715 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:48.443716 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:48.443765 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 15:02:48.443840 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 15:02:48.443951 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 15:02:48.444087 10013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.435804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60408 (local address 127.11.8.124:36135)
0806 15:02:48.435966 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.435968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.435973 (+ 5us) server_negotiation.cc:407] Connection header received
0806 15:02:48.435987 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.435988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.435995 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.436065 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:48.436351 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.437036 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.438056 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.438352 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.438453 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.440774 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:48.440775 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:48.440776 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:48.440782 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:48.442872 (+ 2090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.443217 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.443218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.443218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.443273 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.443483 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.443484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.443485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.443591 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:48.443712 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:48.443881 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 15:02:48.443958 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":71,"threads_started":1}
Aug 06 15:02:48 dist-test-slave-68xk krb5kdc[9699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028568, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.124@KRBTEST.COM for kudu/127.11.8.125@KRBTEST.COM
I20260806 15:02:48.446532 9811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3eec0cf1151414589bc0a48cd4362b6: Bootstrap starting.
I20260806 15:02:48.447388 9811 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3eec0cf1151414589bc0a48cd4362b6: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:48.447734 9811 log.cc:824] T 00000000000000000000000000000000 P b3eec0cf1151414589bc0a48cd4362b6: Log is configured to *not* fsync() on all Append() calls
Aug 06 15:02:48 dist-test-slave-68xk krb5kdc[9699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:02:48.448630 9811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3eec0cf1151414589bc0a48cd4362b6: No bootstrap required, opened a new log
I20260806 15:02:48.449829 10010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.441832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:35777 (local address 127.0.0.1:51862)
0806 15:02:48.441868 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:48.441879 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:48.441915 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:48.442259 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:48.442260 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:48.442760 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:48.442797 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:48.442799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.443700 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.443701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:48.444509 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:48.444512 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.445003 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.445004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:48.445058 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.445464 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:02:48.445469 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:48.446784 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:48.448595 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:48.448596 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:48.448597 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:48.448829 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:48.449069 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:48.449070 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:48.449070 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:48.449150 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:48.449506 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:48.449507 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:48.449553 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 15:02:48.449630 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 15:02:48.449716 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260806 15:02:48.450141 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.442076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51862 (local address 127.11.8.125:35777)
0806 15:02:48.442129 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.442131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.442140 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:02:48.442171 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.442172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.442180 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.442252 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:48.442868 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.443609 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.444610 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.444894 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.444939 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.446876 (+ 1937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:48.446878 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:48.446878 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:48.446886 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:48.448507 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.448913 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.448915 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.448915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.448974 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.449271 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.449272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.449273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.449416 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:48.449479 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:48.449950 (+ 471us) server_negotiation.cc:299] Negotiation successful
0806 15:02:48.450028 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 15:02:48.450582 10006 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36135 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:02:48.450703 9811 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3eec0cf1151414589bc0a48cd4362b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3eec0cf1151414589bc0a48cd4362b6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40203 } } peers { permanent_uuid: "49436d0f114b43f1ac175c84c2f8b98e" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 35777 } } peers { permanent_uuid: "e1c5a048f185471f99d19de7aec1c17b" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36135 } }
I20260806 15:02:48.450832 9811 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3eec0cf1151414589bc0a48cd4362b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3eec0cf1151414589bc0a48cd4362b6, State: Initialized, Role: FOLLOWER
I20260806 15:02:48.451408 9811 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3eec0cf1151414589bc0a48cd4362b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3eec0cf1151414589bc0a48cd4362b6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40203 } } peers { permanent_uuid: "49436d0f114b43f1ac175c84c2f8b98e" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 35777 } } peers { permanent_uuid: "e1c5a048f185471f99d19de7aec1c17b" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36135 } }
I20260806 15:02:48.451541 9814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.442852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54158 (local address 127.11.8.126:40203)
0806 15:02:48.442889 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.442890 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.442935 (+ 45us) server_negotiation.cc:407] Connection header received
0806 15:02:48.443030 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.443031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.443040 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.443067 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:48.443653 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.444330 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.445270 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.445496 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.445535 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.448658 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:48.448659 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:48.448660 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:48.448667 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:48.450312 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.450725 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.450726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.450727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.450781 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.451026 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.451026 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.451027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.451135 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:48.451313 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:48.451350 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 15:02:48.451419 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:02:48.451747 10015 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3eec0cf1151414589bc0a48cd4362b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3eec0cf1151414589bc0a48cd4362b6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40203 } } peers { permanent_uuid: "49436d0f114b43f1ac175c84c2f8b98e" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 35777 } } peers { permanent_uuid: "e1c5a048f185471f99d19de7aec1c17b" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36135 } } }
I20260806 15:02:48.451895 10015 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3eec0cf1151414589bc0a48cd4362b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:02:48.451817 9811 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3eec0cf1151414589bc0a48cd4362b6 [sys.catalog]: configured and running, proceeding with master startup.
Aug 06 15:02:48 dist-test-slave-68xk krb5kdc[9699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.125@KRBTEST.COM
Aug 06 15:02:48 dist-test-slave-68xk krb5kdc[9699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028568, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.124@KRBTEST.COM for kudu/127.11.8.124@KRBTEST.COM
I20260806 15:02:48.454612 9716 master_runner.cc:186] Error getting master registration for 127.11.8.125:35777: OK, instance_id { permanent_uuid: "49436d0f114b43f1ac175c84c2f8b98e" instance_seqno: 1786028568259143 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 15:02:48.457736 9716 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 15:02:48.454609 10027 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b3eec0cf1151414589bc0a48cd4362b6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:02:48.457824 10027 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:02:48.460440 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.452162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51876 (local address 127.11.8.125:35777)
0806 15:02:48.452201 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.452203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.452217 (+ 14us) server_negotiation.cc:407] Connection header received
0806 15:02:48.452235 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.452236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.452243 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.452295 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:48.452756 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.453191 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.454103 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.454391 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.454471 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.457750 (+ 3279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:48.457751 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:48.457752 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:48.457759 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:48.459289 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.459663 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.459665 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.459665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.459727 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.459936 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.459937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.459937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.460049 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:48.460222 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:48.460261 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 15:02:48.460332 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 15:02:48.460707 10013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.450894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60416 (local address 127.11.8.124:36135)
0806 15:02:48.451883 (+ 989us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.451886 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.451894 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:02:48.451914 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.451915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.451924 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.453564 (+ 1640us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:48.453579 (+ 15us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.453996 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.455143 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.455437 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.455475 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.457642 (+ 2167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:48.457643 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:48.457643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:48.457650 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:48.459576 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.459944 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.459945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.459945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.459989 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.460159 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.460159 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.460160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.460324 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:48.460368 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:48.460540 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 15:02:48.460613 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":969}
I20260806 15:02:48.461063 10010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.450830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36135 (local address 127.0.0.1:60416)
0806 15:02:48.450871 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:48.450881 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:48.450917 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:48.453354 (+ 2437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:48.453355 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:48.453486 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:48.453513 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:48.453515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.454081 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.454082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:48.455053 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:48.455056 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.455540 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.455541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:48.455584 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.455967 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:02:48.455972 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:48.457549 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:48.459676 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:48.459676 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:48.459677 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:48.459868 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:48.460053 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:48.460053 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:48.460054 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:48.460099 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:48.460411 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:48.460411 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:48.460444 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 15:02:48.460894 (+ 450us) client_negotiation.cc:240] Negotiation successful
0806 15:02:48.460978 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 15:02:48.461711 11297 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:02:48.462714 10006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1c5a048f185471f99d19de7aec1c17b: Bootstrap starting.
I20260806 15:02:48.462755 9771 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b3eec0cf1151414589bc0a48cd4362b6"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54158
I20260806 15:02:48.462913 9771 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3eec0cf1151414589bc0a48cd4362b6 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 15:02:48.462970 9771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3eec0cf1151414589bc0a48cd4362b6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:48.463455 10006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1c5a048f185471f99d19de7aec1c17b: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:48.463780 10006 log.cc:824] T 00000000000000000000000000000000 P e1c5a048f185471f99d19de7aec1c17b: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:48.463946 9771 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3eec0cf1151414589bc0a48cd4362b6 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3eec0cf1151414589bc0a48cd4362b6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40203 } } peers { permanent_uuid: "49436d0f114b43f1ac175c84c2f8b98e" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 35777 } } peers { permanent_uuid: "e1c5a048f185471f99d19de7aec1c17b" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36135 } }
I20260806 15:02:48.464221 9771 leader_election.cc:288] T 00000000000000000000000000000000 P b3eec0cf1151414589bc0a48cd4362b6 [CANDIDATE]: Term 1 election: Requested vote from peers 49436d0f114b43f1ac175c84c2f8b98e (127.11.8.125:35777), e1c5a048f185471f99d19de7aec1c17b (127.11.8.124:36135)
I20260806 15:02:48.464625 10006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1c5a048f185471f99d19de7aec1c17b: No bootstrap required, opened a new log
I20260806 15:02:48.466939 10006 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1c5a048f185471f99d19de7aec1c17b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3eec0cf1151414589bc0a48cd4362b6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40203 } } peers { permanent_uuid: "49436d0f114b43f1ac175c84c2f8b98e" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 35777 } } peers { permanent_uuid: "e1c5a048f185471f99d19de7aec1c17b" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36135 } }
I20260806 15:02:48.467087 10006 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1c5a048f185471f99d19de7aec1c17b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1c5a048f185471f99d19de7aec1c17b, State: Initialized, Role: FOLLOWER
I20260806 15:02:48.467177 10006 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1c5a048f185471f99d19de7aec1c17b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3eec0cf1151414589bc0a48cd4362b6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40203 } } peers { permanent_uuid: "49436d0f114b43f1ac175c84c2f8b98e" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 35777 } } peers { permanent_uuid: "e1c5a048f185471f99d19de7aec1c17b" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36135 } }
I20260806 15:02:48.467515 10006 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1c5a048f185471f99d19de7aec1c17b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:48.467586 9971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b3eec0cf1151414589bc0a48cd4362b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e1c5a048f185471f99d19de7aec1c17b"
I20260806 15:02:48.468631 9971 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1c5a048f185471f99d19de7aec1c17b [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:48.470103 9971 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e1c5a048f185471f99d19de7aec1c17b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3eec0cf1151414589bc0a48cd4362b6 in term 1.
I20260806 15:02:48.470656 9733 leader_election.cc:302] T 00000000000000000000000000000000 P b3eec0cf1151414589bc0a48cd4362b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3eec0cf1151414589bc0a48cd4362b6, e1c5a048f185471f99d19de7aec1c17b; no voters:
I20260806 15:02:48.470841 10015 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3eec0cf1151414589bc0a48cd4362b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:48.471032 10015 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3eec0cf1151414589bc0a48cd4362b6 [term 1 LEADER]: Becoming Leader. State: Replica: b3eec0cf1151414589bc0a48cd4362b6, State: Running, Role: LEADER
I20260806 15:02:48.471122 10015 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3eec0cf1151414589bc0a48cd4362b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3eec0cf1151414589bc0a48cd4362b6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40203 } } peers { permanent_uuid: "49436d0f114b43f1ac175c84c2f8b98e" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 35777 } } peers { permanent_uuid: "e1c5a048f185471f99d19de7aec1c17b" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36135 } }
I20260806 15:02:48.471565 10015 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3eec0cf1151414589bc0a48cd4362b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3eec0cf1151414589bc0a48cd4362b6. Latest consensus state: current_term: 1 leader_uuid: "b3eec0cf1151414589bc0a48cd4362b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3eec0cf1151414589bc0a48cd4362b6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40203 } } peers { permanent_uuid: "49436d0f114b43f1ac175c84c2f8b98e" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 35777 } } peers { permanent_uuid: "e1c5a048f185471f99d19de7aec1c17b" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36135 } } }
I20260806 15:02:48.471635 10015 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3eec0cf1151414589bc0a48cd4362b6 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:48.472062 10039 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1c5a048f185471f99d19de7aec1c17b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3eec0cf1151414589bc0a48cd4362b6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40203 } } peers { permanent_uuid: "49436d0f114b43f1ac175c84c2f8b98e" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 35777 } } peers { permanent_uuid: "e1c5a048f185471f99d19de7aec1c17b" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36135 } } }
I20260806 15:02:48.472205 10039 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1c5a048f185471f99d19de7aec1c17b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:02:48.472632 10059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:02:48.473086 10059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:02:48.474493 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.465686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51890 (local address 127.11.8.125:35777)
0806 15:02:48.465724 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.465726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.465737 (+ 11us) server_negotiation.cc:407] Connection header received
0806 15:02:48.465835 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.465836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.465846 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.465876 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:48.466692 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.467385 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.468557 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.468868 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.468941 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.470770 (+ 1829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:48.470771 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:48.470772 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:48.470778 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:48.472905 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.473338 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.473339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.473339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.473404 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.473626 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.473627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.473627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.473763 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:48.473844 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:48.474287 (+ 443us) server_negotiation.cc:299] Negotiation successful
0806 15:02:48.474373 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 15:02:48.474553 9971 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e1c5a048f185471f99d19de7aec1c17b [term 1 FOLLOWER]: Refusing update from remote peer b3eec0cf1151414589bc0a48cd4362b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:02:48.475138 10015 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b3eec0cf1151414589bc0a48cd4362b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1c5a048f185471f99d19de7aec1c17b" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:02:48.473183 9917 master_runner.cc:186] Error getting master registration for 127.11.8.125:35777: OK, instance_id { permanent_uuid: "49436d0f114b43f1ac175c84c2f8b98e" instance_seqno: 1786028568259143 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 15:02:48.475447 9917 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 15:02:48.476555 10039 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1c5a048f185471f99d19de7aec1c17b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3eec0cf1151414589bc0a48cd4362b6. Latest consensus state: current_term: 1 leader_uuid: "b3eec0cf1151414589bc0a48cd4362b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3eec0cf1151414589bc0a48cd4362b6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40203 } } peers { permanent_uuid: "49436d0f114b43f1ac175c84c2f8b98e" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 35777 } } peers { permanent_uuid: "e1c5a048f185471f99d19de7aec1c17b" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36135 } } }
I20260806 15:02:48.476670 10039 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1c5a048f185471f99d19de7aec1c17b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:02:48.477808 10057 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3eec0cf1151414589bc0a48cd4362b6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b3eec0cf1151414589bc0a48cd4362b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3eec0cf1151414589bc0a48cd4362b6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40203 } } peers { permanent_uuid: "49436d0f114b43f1ac175c84c2f8b98e" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 35777 } } peers { permanent_uuid: "e1c5a048f185471f99d19de7aec1c17b" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36135 } } }
I20260806 15:02:48.477975 10057 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3eec0cf1151414589bc0a48cd4362b6 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:48.477973 10039 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1c5a048f185471f99d19de7aec1c17b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b3eec0cf1151414589bc0a48cd4362b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3eec0cf1151414589bc0a48cd4362b6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40203 } } peers { permanent_uuid: "49436d0f114b43f1ac175c84c2f8b98e" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 35777 } } peers { permanent_uuid: "e1c5a048f185471f99d19de7aec1c17b" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36135 } } }
I20260806 15:02:48.478061 10039 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1c5a048f185471f99d19de7aec1c17b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:02:48.478140 10062 mvcc.cc:205] Tried to move back new op lower bound from 7315573016469573632 to 7315573016458481664. Current Snapshot: MvccSnapshot[applied={T|T < 7315573016469573632}]
I20260806 15:02:48.478977 10059 catalog_manager.cc:1347] Generated new cluster ID: ce68d3a4611f43a9a8151ba63f64414e
I20260806 15:02:48.479076 10059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 15:02:48.482455 9814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.474620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54168 (local address 127.11.8.126:40203)
0806 15:02:48.474662 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.474664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.474789 (+ 125us) server_negotiation.cc:407] Connection header received
0806 15:02:48.474860 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.474861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.474872 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.474917 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:48.475593 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.476274 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.477212 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.477498 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.477574 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.478962 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:48.478963 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:48.478963 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:48.478970 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:48.480815 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.481236 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.481237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.481237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.481306 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.481531 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.481532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.481532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.481670 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:48.481756 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:48.482253 (+ 497us) server_negotiation.cc:299] Negotiation successful
0806 15:02:48.482328 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 15:02:48.485674 10059 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:02:48.486366 10059 catalog_manager.cc:1504] Loading token signing keys...
I20260806 15:02:48.489550 10013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.477362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60430 (local address 127.11.8.124:36135)
0806 15:02:48.477399 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.477401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.482154 (+ 4753us) server_negotiation.cc:407] Connection header received
0806 15:02:48.482180 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.482181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.482192 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.482251 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:48.483189 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.483903 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.484780 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.485083 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.485155 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.486650 (+ 1495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:48.486651 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:48.486652 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:48.486658 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:48.488316 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.488719 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.488720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.488721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.488781 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.488982 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.488983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.488983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.489103 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:48.489207 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:48.489388 (+ 181us) server_negotiation.cc:299] Negotiation successful
0806 15:02:48.489462 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 15:02:48.490123 10056 catalog_manager.cc:1259] Loaded cluster ID: ce68d3a4611f43a9a8151ba63f64414e
I20260806 15:02:48.490172 10056 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e1c5a048f185471f99d19de7aec1c17b: loading cluster ID for follower catalog manager: success
I20260806 15:02:48.491350 10056 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e1c5a048f185471f99d19de7aec1c17b: acquiring CA information for follower catalog manager: success
W20260806 15:02:48.491479 10056 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260806 15:02:48.493026 10059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3eec0cf1151414589bc0a48cd4362b6: Generated new TSK 0
I20260806 15:02:48.493201 10059 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 15:02:48.496604 9908 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36135 } attempt: 3
Aug 06 15:02:48 dist-test-slave-68xk krb5kdc[9699](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028568, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.125@KRBTEST.COM for kudu/127.11.8.124@KRBTEST.COM
I20260806 15:02:48.505079 10013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.497071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60436 (local address 127.11.8.124:36135)
0806 15:02:48.497107 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.497109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.497116 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:02:48.497137 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.497137 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.497147 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.497191 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:48.497554 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.498263 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.499359 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.499669 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.499730 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.501849 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:48.501851 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:48.501851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:48.501859 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:48.503689 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.504134 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.504135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.504136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.504194 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:48.504408 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:48.504409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:48.504409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:48.504532 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:48.504590 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:48.504889 (+ 299us) server_negotiation.cc:299] Negotiation successful
0806 15:02:48.504963 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:02:48.505285 9910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.496860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36135 (local address 127.0.0.1:60436)
0806 15:02:48.496942 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:48.496953 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:48.496990 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:48.497234 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:48.497235 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:48.497419 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:48.497471 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:48.497473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.498390 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.498391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:48.499252 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:48.499255 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.499780 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.499781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:48.499836 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.500263 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:02:48.500269 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:48.501757 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:48.503809 (+ 2052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:48.503810 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:48.503811 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:48.504064 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:48.504271 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:48.504272 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:48.504273 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:48.504336 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:48.504613 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:48.504614 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:48.504659 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 15:02:48.504728 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 15:02:48.505191 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 15:02:48.506917 9908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49436d0f114b43f1ac175c84c2f8b98e: Bootstrap starting.
I20260806 15:02:48.507651 9908 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49436d0f114b43f1ac175c84c2f8b98e: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:48.507975 9908 log.cc:824] T 00000000000000000000000000000000 P 49436d0f114b43f1ac175c84c2f8b98e: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:48.508916 9908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49436d0f114b43f1ac175c84c2f8b98e: No bootstrap required, opened a new log
I20260806 15:02:48.510959 9908 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49436d0f114b43f1ac175c84c2f8b98e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3eec0cf1151414589bc0a48cd4362b6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40203 } } peers { permanent_uuid: "49436d0f114b43f1ac175c84c2f8b98e" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 35777 } } peers { permanent_uuid: "e1c5a048f185471f99d19de7aec1c17b" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36135 } }
I20260806 15:02:48.511102 9908 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49436d0f114b43f1ac175c84c2f8b98e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49436d0f114b43f1ac175c84c2f8b98e, State: Initialized, Role: FOLLOWER
I20260806 15:02:48.511186 9908 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49436d0f114b43f1ac175c84c2f8b98e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3eec0cf1151414589bc0a48cd4362b6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40203 } } peers { permanent_uuid: "49436d0f114b43f1ac175c84c2f8b98e" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 35777 } } peers { permanent_uuid: "e1c5a048f185471f99d19de7aec1c17b" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36135 } }
I20260806 15:02:48.511503 9908 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49436d0f114b43f1ac175c84c2f8b98e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:48.511592 9870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49436d0f114b43f1ac175c84c2f8b98e [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:48.512563 9870 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 49436d0f114b43f1ac175c84c2f8b98e [term 1 FOLLOWER]: Refusing update from remote peer b3eec0cf1151414589bc0a48cd4362b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:02:48.512768 9871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b3eec0cf1151414589bc0a48cd4362b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "49436d0f114b43f1ac175c84c2f8b98e"
I20260806 15:02:48.513371 9871 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 49436d0f114b43f1ac175c84c2f8b98e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3eec0cf1151414589bc0a48cd4362b6 in term 1.
I20260806 15:02:48.513727 10015 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b3eec0cf1151414589bc0a48cd4362b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49436d0f114b43f1ac175c84c2f8b98e" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 35777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:02:48.516301 10091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49436d0f114b43f1ac175c84c2f8b98e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3eec0cf1151414589bc0a48cd4362b6. Latest consensus state: current_term: 1 leader_uuid: "b3eec0cf1151414589bc0a48cd4362b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3eec0cf1151414589bc0a48cd4362b6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40203 } } peers { permanent_uuid: "49436d0f114b43f1ac175c84c2f8b98e" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 35777 } } peers { permanent_uuid: "e1c5a048f185471f99d19de7aec1c17b" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36135 } } }
I20260806 15:02:48.516374 10091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49436d0f114b43f1ac175c84c2f8b98e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:02:48.516453 10094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49436d0f114b43f1ac175c84c2f8b98e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b3eec0cf1151414589bc0a48cd4362b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3eec0cf1151414589bc0a48cd4362b6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40203 } } peers { permanent_uuid: "49436d0f114b43f1ac175c84c2f8b98e" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 35777 } } peers { permanent_uuid: "e1c5a048f185471f99d19de7aec1c17b" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36135 } } }
I20260806 15:02:48.516505 10094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49436d0f114b43f1ac175c84c2f8b98e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:02:48.511479 10074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49436d0f114b43f1ac175c84c2f8b98e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3eec0cf1151414589bc0a48cd4362b6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40203 } } peers { permanent_uuid: "49436d0f114b43f1ac175c84c2f8b98e" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 35777 } } peers { permanent_uuid: "e1c5a048f185471f99d19de7aec1c17b" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36135 } } }
I20260806 15:02:48.517093 10074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49436d0f114b43f1ac175c84c2f8b98e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:02:48.517942 10088 catalog_manager.cc:1259] Loaded cluster ID: ce68d3a4611f43a9a8151ba63f64414e
I20260806 15:02:48.518029 10088 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 49436d0f114b43f1ac175c84c2f8b98e: loading cluster ID for follower catalog manager: success
I20260806 15:02:48.519200 10088 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 49436d0f114b43f1ac175c84c2f8b98e: acquiring CA information for follower catalog manager: success
I20260806 15:02:48.519574 10088 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 49436d0f114b43f1ac175c84c2f8b98e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 15:02:48.531567 9914 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:02:48.530023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51904 (local address 127.11.8.125:35777)
0806 15:02:48.530064 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.530066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.530926 (+ 860us) server_negotiation.cc:407] Connection header received
0806 15:02:48.531025 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.531026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.531038 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.531067 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:48.531508 (+ 441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51904: BlockingRecv error: recv got EOF from 127.0.0.1:51904 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 15:02:48.532464 9814 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:02:48.530453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54184 (local address 127.11.8.126:40203)
0806 15:02:48.530493 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.530495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.531953 (+ 1458us) server_negotiation.cc:407] Connection header received
0806 15:02:48.532048 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.532049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.532059 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.532089 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:48.532401 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54184: BlockingRecv error: recv got EOF from 127.0.0.1:54184 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 15:02:48.533293 10013 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:02:48.530788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60452 (local address 127.11.8.124:36135)
0806 15:02:48.530830 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.530832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.532789 (+ 1957us) server_negotiation.cc:407] Connection header received
0806 15:02:48.532884 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.532885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.532895 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.532924 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:48.533233 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60452: BlockingRecv error: recv got EOF from 127.0.0.1:60452 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 15:02:48.538493 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.534610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51918 (local address 127.11.8.125:35777)
0806 15:02:48.534643 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.534644 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.535220 (+ 576us) server_negotiation.cc:407] Connection header received
0806 15:02:48.535344 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.535345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.535379 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.535422 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:02:48.535619 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.536375 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.537523 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.537816 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.537857 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.538087 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:02:48.538219 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:02:48.538270 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:48.538383 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 15:02:48.538402 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 15:02:48.541841 10013 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:02:48.541120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60466 (local address 127.11.8.124:36135)
0806 15:02:48.541161 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.541163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.541296 (+ 133us) server_negotiation.cc:407] Connection header received
0806 15:02:48.541403 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.541404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.541422 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.541452 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:48.541787 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60466: BlockingRecv error: recv got EOF from 127.0.0.1:60466 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 15:02:48.547302 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.544059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51934 (local address 127.11.8.125:35777)
0806 15:02:48.544092 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.544094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.544326 (+ 232us) server_negotiation.cc:407] Connection header received
0806 15:02:48.544415 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.544415 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.544424 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.544450 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:02:48.544617 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.545358 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.546530 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.546817 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.546856 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.547047 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:02:48.547077 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:02:48.547104 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:48.547218 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 15:02:48.547237 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 15:02:48.550482 10013 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:02:48.549739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60470 (local address 127.11.8.124:36135)
0806 15:02:48.549774 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.549776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.549955 (+ 179us) server_negotiation.cc:407] Connection header received
0806 15:02:48.550052 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.550052 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.550063 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.550091 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:48.550435 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60470: BlockingRecv error: recv got EOF from 127.0.0.1:60470 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 15:02:48.556864 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.553677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51940 (local address 127.11.8.125:35777)
0806 15:02:48.553717 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.553719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.553932 (+ 213us) server_negotiation.cc:407] Connection header received
0806 15:02:48.554020 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.554021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.554029 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.554056 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:02:48.554225 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.554967 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.556099 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.556395 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.556433 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.556622 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:02:48.556655 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:02:48.556682 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:48.556785 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 15:02:48.556800 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 15:02:48.559952 10013 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:02:48.559257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60486 (local address 127.11.8.124:36135)
0806 15:02:48.559294 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.559295 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.559466 (+ 171us) server_negotiation.cc:407] Connection header received
0806 15:02:48.559557 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.559558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.559567 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.559596 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:48.559901 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60486: BlockingRecv error: recv got EOF from 127.0.0.1:60486 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 15:02:48.568300 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.565134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51946 (local address 127.11.8.125:35777)
0806 15:02:48.565171 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.565172 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.565404 (+ 232us) server_negotiation.cc:407] Connection header received
0806 15:02:48.565497 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.565498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.565507 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.565534 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:02:48.565692 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.566392 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.567517 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.567803 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.567846 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.568040 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:02:48.568071 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:02:48.568098 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:48.568201 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 15:02:48.568228 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 15:02:48.571712 10013 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:02:48.570907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60494 (local address 127.11.8.124:36135)
0806 15:02:48.570944 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.570946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.571182 (+ 236us) server_negotiation.cc:407] Connection header received
0806 15:02:48.571295 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.571296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.571307 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.571336 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:48.571664 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60494: BlockingRecv error: recv got EOF from 127.0.0.1:60494 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 15:02:48.584233 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.580939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51956 (local address 127.11.8.125:35777)
0806 15:02:48.580976 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.580978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.581221 (+ 243us) server_negotiation.cc:407] Connection header received
0806 15:02:48.581321 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.581322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.581331 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.581359 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:02:48.581525 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.582257 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.583439 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.583730 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.583771 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.583964 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:02:48.583997 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:02:48.584024 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:48.584123 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 15:02:48.584140 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 15:02:48.587639 10013 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:02:48.586811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60506 (local address 127.11.8.124:36135)
0806 15:02:48.586850 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.586852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.587083 (+ 231us) server_negotiation.cc:407] Connection header received
0806 15:02:48.587179 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.587180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.587191 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.587234 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:48.587582 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60506: BlockingRecv error: recv got EOF from 127.0.0.1:60506 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 15:02:48.608500 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.605087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51972 (local address 127.11.8.125:35777)
0806 15:02:48.605131 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.605133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.605436 (+ 303us) server_negotiation.cc:407] Connection header received
0806 15:02:48.605536 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.605537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.605547 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.605575 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:02:48.605769 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.606549 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.607677 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.607954 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.608002 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.608222 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:02:48.608254 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:02:48.608285 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:48.608400 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 15:02:48.608421 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 15:02:48.612260 10013 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:02:48.611192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60520 (local address 127.11.8.124:36135)
0806 15:02:48.611246 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.611248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.611654 (+ 406us) server_negotiation.cc:407] Connection header received
0806 15:02:48.611761 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.611762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.611776 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.611806 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:48.612190 (+ 384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60520: BlockingRecv error: recv got EOF from 127.0.0.1:60520 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 15:02:48.649446 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.646004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51986 (local address 127.11.8.125:35777)
0806 15:02:48.646053 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.646055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.646359 (+ 304us) server_negotiation.cc:407] Connection header received
0806 15:02:48.646510 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.646511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.646523 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.646557 (+ 34us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:02:48.646785 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.647514 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.648640 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.648920 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.648965 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.649169 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:02:48.649211 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:02:48.649242 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:48.649349 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 15:02:48.649367 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 15:02:48.653128 10013 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:02:48.652183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60534 (local address 127.11.8.124:36135)
0806 15:02:48.652226 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.652228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.652422 (+ 194us) server_negotiation.cc:407] Connection header received
0806 15:02:48.652520 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.652521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.652534 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.652564 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:48.653074 (+ 510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60534: BlockingRecv error: recv got EOF from 127.0.0.1:60534 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 15:02:48.721151 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.718872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51988 (local address 127.11.8.125:35777)
0806 15:02:48.718979 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.718981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.719017 (+ 36us) server_negotiation.cc:407] Connection header received
0806 15:02:48.719100 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.719101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.719111 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.719168 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:02:48.719297 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.719775 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.720481 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.720656 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.720719 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.720917 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:02:48.720944 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:02:48.720983 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:48.721039 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 15:02:48.721054 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260806 15:02:48.723585 10013 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:02:48.722980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60548 (local address 127.11.8.124:36135)
0806 15:02:48.723062 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.723064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.723117 (+ 53us) server_negotiation.cc:407] Connection header received
0806 15:02:48.723165 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.723166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.723175 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.723222 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:48.723524 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60548: BlockingRecv error: recv got EOF from 127.0.0.1:60548 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 15:02:48.855433 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:48.852966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51994 (local address 127.11.8.125:35777)
0806 15:02:48.853079 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.853081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.853176 (+ 95us) server_negotiation.cc:407] Connection header received
0806 15:02:48.853238 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.853240 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.853250 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.853304 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:02:48.853429 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.853917 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.854741 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:48.854959 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:48.855021 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:48.855227 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:02:48.855256 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:02:48.855296 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:48.855354 (+ 58us) server_negotiation.cc:299] Negotiation successful
0806 15:02:48.855370 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260806 15:02:48.857863 10013 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:02:48.857193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60562 (local address 127.11.8.124:36135)
0806 15:02:48.857259 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:48.857261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:48.857270 (+ 9us) server_negotiation.cc:407] Connection header received
0806 15:02:48.857332 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:48.857333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:48.857343 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:48.857393 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:48.857772 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60562: BlockingRecv error: recv got EOF from 127.0.0.1:60562 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 15:02:49.059909 10224 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3eec0cf1151414589bc0a48cd4362b6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b3eec0cf1151414589bc0a48cd4362b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3eec0cf1151414589bc0a48cd4362b6" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 40203 } } peers { permanent_uuid: "49436d0f114b43f1ac175c84c2f8b98e" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 35777 } } peers { permanent_uuid: "e1c5a048f185471f99d19de7aec1c17b" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36135 } } }
I20260806 15:02:49.060077 10224 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3eec0cf1151414589bc0a48cd4362b6 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:49.118131 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:49.115119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52000 (local address 127.11.8.125:35777)
0806 15:02:49.115215 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:49.115217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:49.115230 (+ 13us) server_negotiation.cc:407] Connection header received
0806 15:02:49.115305 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:49.115306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:49.115317 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:49.115386 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:02:49.115532 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:49.116078 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:49.117385 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:49.117674 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:49.117741 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:49.117888 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:02:49.117921 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:02:49.117966 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:49.118019 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 15:02:49.118036 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260806 15:02:49.120899 10013 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:02:49.120149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60578 (local address 127.11.8.124:36135)
0806 15:02:49.120211 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:49.120213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:49.120316 (+ 103us) server_negotiation.cc:407] Connection header received
0806 15:02:49.120387 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:49.120388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:49.120400 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:49.120456 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:49.120844 (+ 388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60578: BlockingRecv error: recv got EOF from 127.0.0.1:60578 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 15:02:49.492120 10056 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e1c5a048f185471f99d19de7aec1c17b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 15:02:49.636749 10251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:49.634142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52014 (local address 127.11.8.125:35777)
0806 15:02:49.634262 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:49.634264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:49.634307 (+ 43us) server_negotiation.cc:407] Connection header received
0806 15:02:49.634376 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:49.634377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:49.634388 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:49.634477 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:02:49.634621 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:49.635245 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:49.636017 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:49.636259 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:49.636320 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:49.636452 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:02:49.636482 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:02:49.636524 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:49.636580 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 15:02:49.636599 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":66,"threads_started":1}
I20260806 15:02:49.641358 10259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:49.638746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60588 (local address 127.11.8.124:36135)
0806 15:02:49.638864 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:49.638865 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:49.638890 (+ 25us) server_negotiation.cc:407] Connection header received
0806 15:02:49.638915 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:49.638916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:49.638969 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:49.639036 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:02:49.639164 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:49.639738 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:49.640521 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:49.640750 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:49.640812 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:49.640974 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:02:49.641103 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:02:49.641167 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:49.641271 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 15:02:49.641292 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":67,"threads_started":1}
I20260806 15:02:49.646463 10267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:49.643359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54188 (local address 127.11.8.126:40203)
0806 15:02:49.643621 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:49.643624 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:49.643634 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:02:49.643719 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:49.643721 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:49.643781 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:49.643874 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:02:49.644289 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:49.644797 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:49.645580 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:49.645797 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:49.645875 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:49.646079 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:02:49.646194 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:02:49.646257 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:49.646332 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 15:02:49.646354 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":71,"threads_started":1}
I20260806 15:02:49.647081 10251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:49.643517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52028 (local address 127.11.8.125:35777)
0806 15:02:49.643562 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:49.643564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:49.643730 (+ 166us) server_negotiation.cc:407] Connection header received
0806 15:02:49.643869 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:49.643871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:49.643882 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:49.644091 (+ 209us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:02:49.644294 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:49.645092 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:49.646167 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:49.646471 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:49.646543 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:49.646804 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:02:49.646840 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:02:49.646894 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:49.646965 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 15:02:49.646990 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 15:02:49.647143 10259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:49.643713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60602 (local address 127.11.8.124:36135)
0806 15:02:49.643810 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:49.643812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:49.643819 (+ 7us) server_negotiation.cc:407] Connection header received
0806 15:02:49.643840 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:49.643841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:49.643854 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:49.643906 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:02:49.644092 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:49.644905 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:49.646165 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:49.646525 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:49.646576 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:49.646825 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:02:49.646860 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:02:49.646914 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:49.647016 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 15:02:49.647042 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 15:02:49.648607 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 9716
I20260806 15:02:49.662256 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 9819
I20260806 15:02:49.672174 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 9917
2026-08-06T15:02:49Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3122 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 15:02:49 dist-test-slave-68xk krb5kdc[10273](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 15:02:49 dist-test-slave-68xk krb5kdc[10273](info): set up 2 sockets
Aug 06 15:02:49 dist-test-slave-68xk krb5kdc[10273](info): commencing operation
krb5kdc: starting...
W20260806 15:02:50.873919 11297 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/krb5kdc/test-user.keytab.
Aug 06 15:02:50 dist-test-slave-68xk krb5kdc[10273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T15:02:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T15:02:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.11.8.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
Entry for principal kudu/127.11.8.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab.
I20260806 15:02:51.001237 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:44917
--webserver_interface=127.11.8.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:33275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:44917,127.11.8.125:41549,127.11.8.124:36887
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 15:02:51.089550 10287 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 15:02:51.089768 10287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:51.089787 10287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:51.089839 10287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:51.090991 10287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:51.091032 10287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:51.091046 10287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:51.091058 10287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:51.091070 10287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:51.092592 10287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:33275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:44917,127.11.8.125:41549,127.11.8.124:36887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.126.keytab
--principal=kudu/127.11.8.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.126:44917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.11.8.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:51.092857 10287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:51.093101 10287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:51.098021 10293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:51.098096 10287 server_base.cc:1034] running on GCE node
W20260806 15:02:51.098047 10295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:51.098047 10292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:51.098582 10287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:51.098920 10287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:51.100075 10287 hybrid_clock.cc:648] HybridClock initialized: now 1786028571100060 us; error 43 us; skew 500 ppm
Aug 06 15:02:51 dist-test-slave-68xk krb5kdc[10273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028571, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:51.102628 10287 init.cc:377] Logged in from keytab as kudu/127.11.8.126@KRBTEST.COM (short username kudu)
I20260806 15:02:51.103077 10287 webserver.cc:460] Webserver started at http://127.11.8.126:44435/ using document root <none> and password file <none>
I20260806 15:02:51.103340 10287 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:51.103412 10287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:51.103569 10287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:51.104626 10287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-0/data/instance:
uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420"
format_stamp: "Formatted at 2026-08-06 15:02:51 on dist-test-slave-68xk"
server_key: "1f6e7339c11d0160f980434be2d82604"
server_key_iv: "2e725bcacbae0c4ac40e2d104a18eb9e"
server_key_version: "encryptionkey@0"
I20260806 15:02:51.104998 10287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-0/wal/instance:
uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420"
format_stamp: "Formatted at 2026-08-06 15:02:51 on dist-test-slave-68xk"
server_key: "1f6e7339c11d0160f980434be2d82604"
server_key_iv: "2e725bcacbae0c4ac40e2d104a18eb9e"
server_key_version: "encryptionkey@0"
I20260806 15:02:51.106341 10287 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 15:02:51.107141 10306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:51.107314 10287 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 15:02:51.107422 10287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-0/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-0/wal
uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420"
format_stamp: "Formatted at 2026-08-06 15:02:51 on dist-test-slave-68xk"
server_key: "1f6e7339c11d0160f980434be2d82604"
server_key_iv: "2e725bcacbae0c4ac40e2d104a18eb9e"
server_key_version: "encryptionkey@0"
I20260806 15:02:51.107496 10287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:51.127014 10287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:51.129029 10287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:51.129200 10287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:51.133379 10287 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.126:44917
I20260806 15:02:51.133584 10382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.126:44917 every 8 connection(s)
I20260806 15:02:51.133843 10287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-0/data/info.pb
I20260806 15:02:51.134786 10384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:51.136391 10384 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44917 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:02:51.137084 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 10287
I20260806 15:02:51.137223 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-0/wal/instance
I20260806 15:02:51.137477 11297 external_mini_cluster.cc:1428] Setting key 35445913eb372b4ad3aa6961c8f20c2e
Aug 06 15:02:51 dist-test-slave-68xk krb5kdc[10273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028571, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.11.8.125@KRBTEST.COM; defaulting to no policy
I20260806 15:02:51.147795 10387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.137067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:44917 (local address 127.0.0.1:35294)
0806 15:02:51.137352 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:51.137404 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:51.138145 (+ 741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:51.138458 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:51.138461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:51.138854 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:51.139077 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:51.139083 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.140092 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.140093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:51.141048 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:51.141051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.141764 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.141765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:51.141822 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:51.142389 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:02:51.142398 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:51.144099 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:51.146212 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:51.146219 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:51.146221 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:51.146483 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:51.146764 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:51.146767 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:51.146767 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:51.146855 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:51.147332 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:51.147339 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:51.147400 (+ 61us) client_negotiation.cc:769] Sending connection context
0806 15:02:51.147478 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 15:02:51.147617 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":136,"threads_started":1}
I20260806 15:02:51.147799 10388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.137149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35294 (local address 127.11.8.126:44917)
0806 15:02:51.137351 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:51.137360 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:51.137512 (+ 152us) server_negotiation.cc:407] Connection header received
0806 15:02:51.138299 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:51.138301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:51.138318 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:51.138395 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:51.139201 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.139991 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.141144 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.141642 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.141780 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:51.144193 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:51.144203 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:51.144204 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:51.144212 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:51.146124 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.146568 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.146571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.146571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.146688 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.146955 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.146957 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.146957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.147252 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:51.147348 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:51.147541 (+ 193us) server_negotiation.cc:299] Negotiation successful
0806 15:02:51.147693 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":169,"thread_start_us":71,"threads_started":1}
I20260806 15:02:51.148610 10384 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.125" port: 41549 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:02:51.148908 10387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.148825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:41549 (local address 127.0.0.1:57552)
0806 15:02:51.148854 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:51.148893 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.125:41549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260806 15:02:51.149044 10308 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.8.125:41549: connect: Connection refused (error 111)
I20260806 15:02:51.149523 10387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.149448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:41549 (local address 127.0.0.1:57564)
0806 15:02:51.149477 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:51.149503 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.125:41549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260806 15:02:51.149653 10384 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.125:41549: Network error: Client connection negotiation failed: client connection to 127.11.8.125:41549: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.125.keytab.
Entry for principal kudu/127.11.8.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.125.keytab.
I20260806 15:02:51.165958 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.125:41549
--webserver_interface=127.11.8.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.125.keytab
--principal=kudu/127.11.8.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:33275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:44917,127.11.8.125:41549,127.11.8.124:36887
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:02:51.198894 10384 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.125" port: 41549 } attempt: 1
I20260806 15:02:51.199486 10387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.199336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:41549 (local address 127.0.0.1:57574)
0806 15:02:51.199400 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:51.199447 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.125:41549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 15:02:51.199899 10387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.199803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:41549 (local address 127.0.0.1:57584)
0806 15:02:51.199848 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:51.199879 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.125:41549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 15:02:51.199993 10384 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.125:41549: Network error: Client connection negotiation failed: client connection to 127.11.8.125:41549: connect: Connection refused (error 111)
W20260806 15:02:51.250926 10392 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 15:02:51.251171 10392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:51.251210 10392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:51.251255 10392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:51.252383 10392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:51.252447 10392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:51.252472 10392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:51.252496 10392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:51.252542 10392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:51.254019 10392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:33275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:44917,127.11.8.125:41549,127.11.8.124:36887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.125.keytab
--principal=kudu/127.11.8.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.125:41549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.11.8.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:51.254304 10392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:51.254550 10392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:51.259400 10400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:51.259425 10401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:51.259425 10405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:51.259734 10392 server_base.cc:1034] running on GCE node
I20260806 15:02:51.259928 10392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:51.260207 10392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:51.261389 10392 hybrid_clock.cc:648] HybridClock initialized: now 1786028571261380 us; error 36 us; skew 500 ppm
Aug 06 15:02:51 dist-test-slave-68xk krb5kdc[10273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028571, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:51.264024 10392 init.cc:377] Logged in from keytab as kudu/127.11.8.125@KRBTEST.COM (short username kudu)
I20260806 15:02:51.264472 10392 webserver.cc:460] Webserver started at http://127.11.8.125:36987/ using document root <none> and password file <none>
I20260806 15:02:51.264657 10392 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:51.264698 10392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:51.264788 10392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:51.265769 10392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-1/data/instance:
uuid: "17d80ae85ffd48a596cfa7e588186df7"
format_stamp: "Formatted at 2026-08-06 15:02:51 on dist-test-slave-68xk"
server_key: "b9c969592521a22674c8f1859f7126c3"
server_key_iv: "cfd106da2e89d33f1a8ed46d080683f0"
server_key_version: "encryptionkey@0"
I20260806 15:02:51.266081 10392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-1/wal/instance:
uuid: "17d80ae85ffd48a596cfa7e588186df7"
format_stamp: "Formatted at 2026-08-06 15:02:51 on dist-test-slave-68xk"
server_key: "b9c969592521a22674c8f1859f7126c3"
server_key_iv: "cfd106da2e89d33f1a8ed46d080683f0"
server_key_version: "encryptionkey@0"
I20260806 15:02:51.267462 10392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 15:02:51.268131 10384 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.125" port: 41549 } attempt: 2
I20260806 15:02:51.268253 10418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:51.268437 10392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:02:51.268553 10392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-1/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-1/wal
uuid: "17d80ae85ffd48a596cfa7e588186df7"
format_stamp: "Formatted at 2026-08-06 15:02:51 on dist-test-slave-68xk"
server_key: "b9c969592521a22674c8f1859f7126c3"
server_key_iv: "cfd106da2e89d33f1a8ed46d080683f0"
server_key_version: "encryptionkey@0"
I20260806 15:02:51.268571 10387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.268439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:41549 (local address 127.0.0.1:57590)
0806 15:02:51.268498 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:51.268537 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.125:41549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 15:02:51.268631 10392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:51.268950 10387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.268849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:41549 (local address 127.0.0.1:57592)
0806 15:02:51.268898 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:51.268924 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.125:41549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260806 15:02:51.269059 10384 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.125:41549: Network error: Client connection negotiation failed: client connection to 127.11.8.125:41549: connect: Connection refused (error 111)
I20260806 15:02:51.298096 10392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:51.300594 10392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:51.300770 10392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:51.304845 10392 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.125:41549
I20260806 15:02:51.304909 10487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.125:41549 every 8 connection(s)
I20260806 15:02:51.305295 10392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-1/data/info.pb
I20260806 15:02:51.306357 10488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:51.307957 10488 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44917 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:02:51.311472 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 10392
I20260806 15:02:51.311600 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-1/wal/instance
I20260806 15:02:51.311843 11297 external_mini_cluster.cc:1428] Setting key 93e343730f0b880c5ee2dbafb55b0ce9
Aug 06 15:02:51 dist-test-slave-68xk krb5kdc[10273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028571, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.125@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:02:51.318205 10388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.308643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35296 (local address 127.11.8.126:44917)
0806 15:02:51.308720 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:51.308722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:51.308928 (+ 206us) server_negotiation.cc:407] Connection header received
0806 15:02:51.309365 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:51.309367 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:51.309381 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:51.309448 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:51.310112 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.310718 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.311828 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.312162 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.312240 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:51.314710 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:51.314711 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:51.314711 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:51.314718 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:51.316605 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.317075 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.317077 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.317077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.317153 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.317439 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.317441 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.317441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.317603 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:51.317691 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:51.317979 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 15:02:51.318075 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 15:02:51.318239 10491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.308585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:44917 (local address 127.0.0.1:35296)
0806 15:02:51.308762 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:51.308804 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:51.309275 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:51.309525 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:51.309529 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:51.309782 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:51.310020 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:51.310025 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.310810 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.310811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:51.311746 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:51.311749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.312264 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.312266 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:51.312363 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:51.312886 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:02:51.312894 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:51.314602 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:51.316721 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:51.316729 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:51.316731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:51.316988 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:51.317230 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:51.317231 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:51.317231 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:51.317361 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:51.317689 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:51.317697 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:51.317833 (+ 136us) client_negotiation.cc:769] Sending connection context
0806 15:02:51.317928 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 15:02:51.318058 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":103,"threads_started":1}
I20260806 15:02:51.318881 10488 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.125" port: 41549 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.11.8.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.11.8.124@KRBTEST.COM" created.
Aug 06 15:02:51 dist-test-slave-68xk krb5kdc[10273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028571, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.125@KRBTEST.COM for kudu/127.11.8.125@KRBTEST.COM
I20260806 15:02:51.327816 10491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.319157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:41549 (local address 127.0.0.1:57608)
0806 15:02:51.319399 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:51.319417 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:51.319472 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:51.319656 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:51.319657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:51.319810 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:51.319861 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:51.319862 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.320813 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.320815 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:51.321444 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:51.321446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.321986 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.321987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:51.322059 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:51.322508 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:02:51.322513 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:51.324822 (+ 2309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:51.326663 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:51.326664 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:51.326664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:51.326823 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:51.327102 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:51.327103 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:51.327103 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:51.327164 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:51.327461 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:51.327462 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:51.327501 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 15:02:51.327596 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 15:02:51.327697 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 15:02:51.327881 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.319235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57608 (local address 127.11.8.125:41549)
0806 15:02:51.319414 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:51.319420 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:51.319472 (+ 52us) server_negotiation.cc:407] Connection header received
0806 15:02:51.319536 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:51.319539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:51.319553 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:51.319656 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:51.319955 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.320714 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.321559 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.321888 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.321976 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:51.324909 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:51.324917 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:51.324917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:51.324924 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:51.326587 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.326929 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.326932 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.326933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.326992 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.327232 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.327233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.327233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.327378 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:51.327468 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:51.327631 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 15:02:51.327753 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":146,"thread_start_us":116,"threads_started":1}
I20260806 15:02:51.328454 10488 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36887 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:02:51.328812 10491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.328710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36887 (local address 127.0.0.1:47660)
0806 15:02:51.328750 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:51.328793 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 15:02:51.328987 10420 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36887: connect: Connection refused (error 111)
I20260806 15:02:51.329438 10491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.329343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36887 (local address 127.0.0.1:47676)
0806 15:02:51.329392 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:51.329422 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 15:02:51.329568 10488 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.124:36887: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36887: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.11.8.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.124.keytab.
Entry for principal kudu/127.11.8.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.124.keytab.
I20260806 15:02:51.340960 11297 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskF1cjFt/build/release/bin/kudu
/tmp/dist-test-taskF1cjFt/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.124:36887
--webserver_interface=127.11.8.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.124.keytab
--principal=kudu/127.11.8.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.11.8.84:33275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.11.8.126:44917,127.11.8.125:41549,127.11.8.124:36887
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 15:02:51.360211 10384 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.125" port: 41549 } attempt: 3
Aug 06 15:02:51 dist-test-slave-68xk krb5kdc[10273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028571, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for kudu/127.11.8.125@KRBTEST.COM
I20260806 15:02:51.367585 10387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.360609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:41549 (local address 127.0.0.1:44534)
0806 15:02:51.360703 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:51.360717 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:51.360803 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:51.360967 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:51.360969 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:51.361145 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:51.361212 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:51.361214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.361886 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.361887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:51.362496 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:51.362498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.362912 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.362913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:51.363000 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:51.363484 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:02:51.363489 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:51.364756 (+ 1267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:51.366433 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:51.366435 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:51.366435 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:51.366632 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:51.366870 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:51.366871 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:51.366872 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:51.366940 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:51.367215 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:51.367217 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:51.367282 (+ 65us) client_negotiation.cc:769] Sending connection context
0806 15:02:51.367358 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 15:02:51.367474 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 15:02:51.367584 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.360746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44534 (local address 127.11.8.125:41549)
0806 15:02:51.360817 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:51.360819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:51.360831 (+ 12us) server_negotiation.cc:407] Connection header received
0806 15:02:51.360863 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:51.360864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:51.360879 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:51.360960 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:51.361303 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.361793 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.362601 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.362834 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.362915 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:51.364850 (+ 1935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:51.364852 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:51.364852 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:51.364860 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:51.366300 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.366727 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.366729 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.366729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.366793 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.367013 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.367014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.367014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.367134 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:51.367210 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:51.367397 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 15:02:51.367486 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 15:02:51.368077 10384 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36887 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:02:51.368448 10387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.368344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36887 (local address 127.0.0.1:56458)
0806 15:02:51.368396 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:51.368423 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 15:02:51.368772 10387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.368685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36887 (local address 127.0.0.1:56460)
0806 15:02:51.368728 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:51.368752 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 15:02:51.368868 10384 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.124:36887: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36887: connect: Connection refused (error 111)
I20260806 15:02:51.378769 10488 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36887 } attempt: 1
I20260806 15:02:51.379163 10491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.379040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36887 (local address 127.0.0.1:56464)
0806 15:02:51.379109 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:51.379139 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 15:02:51.379494 10491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.379397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36887 (local address 127.0.0.1:56466)
0806 15:02:51.379445 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:51.379469 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 15:02:51.379635 10488 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.124:36887: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36887: connect: Connection refused (error 111)
I20260806 15:02:51.400008 10384 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36887 } attempt: 1
I20260806 15:02:51.400496 10387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.400337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36887 (local address 127.0.0.1:56476)
0806 15:02:51.400409 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:51.400458 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 15:02:51.400902 10387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.400792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36887 (local address 127.0.0.1:56480)
0806 15:02:51.400841 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:51.400874 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 15:02:51.401010 10384 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.124:36887: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36887: connect: Connection refused (error 111)
W20260806 15:02:51.425981 10497 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 15:02:51.426194 10497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 15:02:51.426213 10497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 15:02:51.426232 10497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 15:02:51.427423 10497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 15:02:51.427469 10497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 15:02:51.427482 10497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 15:02:51.427496 10497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 15:02:51.427508 10497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 15:02:51.429028 10497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.11.8.84:33275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.11.8.126:44917,127.11.8.125:41549,127.11.8.124:36887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/krb5kdc/kudu_127.11.8.124.keytab
--principal=kudu/127.11.8.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.11.8.124:36887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.11.8.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 14:02:25 UTC on ccycloud.quasar-zyippm.root.comops.site
build id 4570387
I20260806 15:02:51.429275 10497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 15:02:51.429529 10497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 15:02:51.434665 10510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 15:02:51.434695 10505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:51.434718 10497 server_base.cc:1034] running on GCE node
W20260806 15:02:51.434665 10513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 15:02:51.435196 10497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 15:02:51.435575 10497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 15:02:51.436722 10497 hybrid_clock.cc:648] HybridClock initialized: now 1786028571436704 us; error 43 us; skew 500 ppm
Aug 06 15:02:51 dist-test-slave-68xk krb5kdc[10273](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028571, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 15:02:51.439353 10497 init.cc:377] Logged in from keytab as kudu/127.11.8.124@KRBTEST.COM (short username kudu)
I20260806 15:02:51.439822 10497 webserver.cc:460] Webserver started at http://127.11.8.124:40095/ using document root <none> and password file <none>
I20260806 15:02:51.440055 10497 fs_manager.cc:359] Metadata directory not provided
I20260806 15:02:51.440124 10497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 15:02:51.440274 10497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 15:02:51.441318 10497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-2/data/instance:
uuid: "63ab3db83fde447ea8a91f73c5fa23dc"
format_stamp: "Formatted at 2026-08-06 15:02:51 on dist-test-slave-68xk"
server_key: "a7268bf8aec1ce3ce79358ee0d16a9cf"
server_key_iv: "f00e9dd60c599f9481172849638c56ca"
server_key_version: "encryptionkey@0"
I20260806 15:02:51.441674 10497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-2/wal/instance:
uuid: "63ab3db83fde447ea8a91f73c5fa23dc"
format_stamp: "Formatted at 2026-08-06 15:02:51 on dist-test-slave-68xk"
server_key: "a7268bf8aec1ce3ce79358ee0d16a9cf"
server_key_iv: "f00e9dd60c599f9481172849638c56ca"
server_key_version: "encryptionkey@0"
I20260806 15:02:51.443151 10497 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 15:02:51.443892 10521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 15:02:51.444101 10497 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 15:02:51.444208 10497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-2/data,/tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-2/wal
uuid: "63ab3db83fde447ea8a91f73c5fa23dc"
format_stamp: "Formatted at 2026-08-06 15:02:51 on dist-test-slave-68xk"
server_key: "a7268bf8aec1ce3ce79358ee0d16a9cf"
server_key_iv: "f00e9dd60c599f9481172849638c56ca"
server_key_version: "encryptionkey@0"
I20260806 15:02:51.444288 10497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 15:02:51.447813 10488 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36887 } attempt: 2
I20260806 15:02:51.448196 10491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.448059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36887 (local address 127.0.0.1:56496)
0806 15:02:51.448116 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:51.448163 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 15:02:51.448522 10491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.448425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36887 (local address 127.0.0.1:56506)
0806 15:02:51.448470 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:51.448494 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 15:02:51.448602 10488 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.11.8.124:36887: Network error: Client connection negotiation failed: client connection to 127.11.8.124:36887: connect: Connection refused (error 111)
I20260806 15:02:51.457865 10497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 15:02:51.460134 10497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 15:02:51.460294 10497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 15:02:51.464442 10497 rpc_server.cc:307] RPC server started. Bound to: 127.11.8.124:36887
I20260806 15:02:51.464479 10588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.11.8.124:36887 every 8 connection(s)
I20260806 15:02:51.464856 10497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-2/data/info.pb
I20260806 15:02:51.465950 10589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 15:02:51.466056 11297 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskF1cjFt/build/release/bin/kudu as pid 10497
I20260806 15:02:51.466187 11297 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskF1cjFt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786028287744255-11297-0/minicluster-data/master-2/wal/instance
I20260806 15:02:51.466509 11297 external_mini_cluster.cc:1428] Setting key 8d0ca1d284ebe416cdb972c4273c83e5
I20260806 15:02:51.468693 10589 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44917 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 15:02:51 dist-test-slave-68xk krb5kdc[10273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.124@KRBTEST.COM
Aug 06 15:02:51 dist-test-slave-68xk krb5kdc[10273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028571, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.124@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:02:51.476197 10384 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36887 } attempt: 2
I20260806 15:02:51.478660 10388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.469455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51406 (local address 127.11.8.126:44917)
0806 15:02:51.469527 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:51.469529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:51.469677 (+ 148us) server_negotiation.cc:407] Connection header received
0806 15:02:51.469739 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:51.469740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:51.469750 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:51.469811 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:51.470347 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.470890 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.471984 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.472279 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.472336 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:51.475076 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:51.475078 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:51.475078 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:51.475084 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:51.477265 (+ 2181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.477738 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.477740 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.477740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.477803 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.478007 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.478008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.478008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.478127 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:51.478215 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:51.478464 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 15:02:51.478553 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 15:02:51.478857 10595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.469352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.126:44917 (local address 127.0.0.1:51406)
0806 15:02:51.469552 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:51.469593 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:51.469690 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:51.469843 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:51.469845 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:51.470168 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:51.470256 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:51.470258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.470988 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.470989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:51.471900 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:51.471903 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.472392 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.472394 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:51.472482 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:51.472936 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:02:51.472950 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:51.474981 (+ 2031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:51.477357 (+ 2376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:51.477369 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:51.477371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:51.477658 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:51.477868 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:51.477869 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:51.477869 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:51.477935 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:51.478208 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:51.478215 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:51.478291 (+ 76us) client_negotiation.cc:769] Sending connection context
0806 15:02:51.478410 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 15:02:51.478592 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":126,"threads_started":1}
I20260806 15:02:51.479043 10593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.467085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56522 (local address 127.11.8.124:36887)
0806 15:02:51.467277 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:51.467292 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:51.467322 (+ 30us) server_negotiation.cc:407] Connection header received
0806 15:02:51.467739 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:51.467746 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:51.467790 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:51.467901 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:51.468799 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.469972 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.471113 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.471573 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.471653 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:51.474488 (+ 2835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:51.474497 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:51.474498 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:51.474505 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:51.476100 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.477266 (+ 1166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.477271 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.477271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.477380 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.477617 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.477618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.477619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.477921 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:51.478059 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:51.478841 (+ 782us) server_negotiation.cc:299] Negotiation successful
0806 15:02:51.478929 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":91,"threads_started":1}
I20260806 15:02:51.479542 10589 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.125" port: 41549 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 15:02:51 dist-test-slave-68xk krb5kdc[10273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028571, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.126@KRBTEST.COM for kudu/127.11.8.124@KRBTEST.COM
Aug 06 15:02:51 dist-test-slave-68xk krb5kdc[10273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028571, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.124@KRBTEST.COM for kudu/127.11.8.125@KRBTEST.COM
Aug 06 15:02:51 dist-test-slave-68xk krb5kdc[10273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.126@KRBTEST.COM
I20260806 15:02:51.486733 10387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.476499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36887 (local address 127.0.0.1:56532)
0806 15:02:51.476553 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:51.476571 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:51.476640 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:51.476932 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:51.476933 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:51.477217 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:51.477264 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:51.477265 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.478221 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.478222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:51.479049 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:51.479052 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.479708 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.479709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:51.479770 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:51.480182 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:02:51.480186 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:51.481767 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:51.484863 (+ 3096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:51.484864 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:51.484865 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:51.485110 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:51.485721 (+ 611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:51.485722 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:51.485722 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:51.485806 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:51.486490 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:51.486492 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:51.486539 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 15:02:51.486589 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 15:02:51.486668 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 15:02:51.487658 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.479889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44536 (local address 127.11.8.125:41549)
0806 15:02:51.479946 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:51.479948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:51.480195 (+ 247us) server_negotiation.cc:407] Connection header received
0806 15:02:51.480277 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:51.480278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:51.480289 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:51.480352 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:51.480635 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.481417 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.482393 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.482730 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.482810 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:51.484811 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:51.484813 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:51.484813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:51.484821 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:51.486349 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.486776 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.486778 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.486778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.486845 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.487073 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.487075 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.487075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.487207 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:51.487288 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:51.487468 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 15:02:51.487555 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 15:02:51.487668 10595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.479786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.125:41549 (local address 127.0.0.1:44536)
0806 15:02:51.479882 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:51.480133 (+ 251us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:51.480208 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:51.480355 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:51.480356 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:51.480502 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:51.480547 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:51.480548 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.481525 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.481527 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:51.482305 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:51.482308 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.482816 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.482817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:51.482893 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:51.483324 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:02:51.483329 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:51.484694 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:51.486494 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:51.486495 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:51.486496 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:51.486680 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:51.486933 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:51.486934 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:51.486935 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:51.486999 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:51.487293 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:51.487294 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:51.487338 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 15:02:51.487424 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 15:02:51.487563 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260806 15:02:51.487865 10597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.476629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56532 (local address 127.11.8.124:36887)
0806 15:02:51.476789 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:51.476791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:51.476801 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:02:51.476823 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:51.476823 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:51.476836 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:51.476922 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:51.477329 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.478110 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.479318 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.479604 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.479702 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:51.481858 (+ 2156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:51.481860 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:51.481860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:51.481868 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:51.484779 (+ 2911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.485192 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.485194 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.485194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.485294 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.485884 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.485885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.485885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.486348 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:51.487540 (+ 1192us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:51.487580 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 15:02:51.487669 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":84,"threads_started":1}
I20260806 15:02:51.488286 10589 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36887 } has no permanent_uuid. Determining permanent_uuid...
I20260806 15:02:51.489297 10388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.479844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51410 (local address 127.11.8.126:44917)
0806 15:02:51.479882 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:51.479884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:51.479977 (+ 93us) server_negotiation.cc:407] Connection header received
0806 15:02:51.480066 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:51.480067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:51.480076 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:51.480123 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:51.480723 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.481177 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.481915 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.482199 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.482233 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:51.485504 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:51.485505 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:51.485506 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:51.485511 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:51.487362 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.488304 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.488306 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.488306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.488367 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.488639 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.488640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.488640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.488752 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:51.488877 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:51.489091 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 15:02:51.489188 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 15:02:51.490042 10384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ea5cc7f1c4a4b6aaaa42d4da276b420: Bootstrap starting.
I20260806 15:02:51.490904 10384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ea5cc7f1c4a4b6aaaa42d4da276b420: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:51.491255 10384 log.cc:824] T 00000000000000000000000000000000 P 7ea5cc7f1c4a4b6aaaa42d4da276b420: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:51.492231 10384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ea5cc7f1c4a4b6aaaa42d4da276b420: No bootstrap required, opened a new log
Aug 06 15:02:51 dist-test-slave-68xk krb5kdc[10273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028571, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.124@KRBTEST.COM for kudu/127.11.8.124@KRBTEST.COM
I20260806 15:02:51.494176 10384 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ea5cc7f1c4a4b6aaaa42d4da276b420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44917 } } peers { permanent_uuid: "17d80ae85ffd48a596cfa7e588186df7" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 41549 } } peers { permanent_uuid: "63ab3db83fde447ea8a91f73c5fa23dc" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36887 } }
I20260806 15:02:51.494293 10384 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ea5cc7f1c4a4b6aaaa42d4da276b420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ea5cc7f1c4a4b6aaaa42d4da276b420, State: Initialized, Role: FOLLOWER
I20260806 15:02:51.494347 10384 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ea5cc7f1c4a4b6aaaa42d4da276b420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44917 } } peers { permanent_uuid: "17d80ae85ffd48a596cfa7e588186df7" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 41549 } } peers { permanent_uuid: "63ab3db83fde447ea8a91f73c5fa23dc" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36887 } }
I20260806 15:02:51.494719 10599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ea5cc7f1c4a4b6aaaa42d4da276b420 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44917 } } peers { permanent_uuid: "17d80ae85ffd48a596cfa7e588186df7" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 41549 } } peers { permanent_uuid: "63ab3db83fde447ea8a91f73c5fa23dc" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36887 } } }
I20260806 15:02:51.494760 10384 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ea5cc7f1c4a4b6aaaa42d4da276b420 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:51.494829 10599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ea5cc7f1c4a4b6aaaa42d4da276b420 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 15:02:51.496503 10613 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7ea5cc7f1c4a4b6aaaa42d4da276b420: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:02:51.496564 10613 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:02:51.497558 10287 master_runner.cc:186] Error getting master registration for 127.11.8.125:41549: OK, instance_id { permanent_uuid: "17d80ae85ffd48a596cfa7e588186df7" instance_seqno: 1786028571303800 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 15:02:51.500087 10287 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 15:02:51.501331 10595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.488573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36887 (local address 127.0.0.1:56536)
0806 15:02:51.488633 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:51.488644 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:51.488692 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:51.488810 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:51.488811 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:51.488950 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:51.488975 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:51.488977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.489662 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.489663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:51.490537 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:51.490540 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.491093 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.491095 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:51.491139 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:51.491571 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:02:51.491577 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:51.492967 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:51.500014 (+ 7047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:51.500016 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:51.500016 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:51.500260 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:51.500502 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:51.500503 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:51.500504 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:51.500576 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:51.500957 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:51.500958 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:51.501008 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 15:02:51.501084 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 15:02:51.501205 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 15:02:51.501413 10597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.488607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56536 (local address 127.11.8.124:36887)
0806 15:02:51.488640 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:51.488642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:51.488677 (+ 35us) server_negotiation.cc:407] Connection header received
0806 15:02:51.488752 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:51.488753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:51.488761 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:51.488809 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:51.489065 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.489574 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.490636 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.490986 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.491049 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:51.493069 (+ 2020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:51.493070 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:51.493070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:51.493077 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:51.499909 (+ 6832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.500351 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.500352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.500353 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.500420 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.500664 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.500665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.500666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.500876 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:51.500952 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:51.501127 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 15:02:51.501281 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":19}
Aug 06 15:02:51 dist-test-slave-68xk krb5kdc[10273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.11.8.125@KRBTEST.COM
I20260806 15:02:51.503116 10589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63ab3db83fde447ea8a91f73c5fa23dc: Bootstrap starting.
I20260806 15:02:51.503906 10589 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63ab3db83fde447ea8a91f73c5fa23dc: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:51.504236 10589 log.cc:824] T 00000000000000000000000000000000 P 63ab3db83fde447ea8a91f73c5fa23dc: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:51.505152 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.490570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44540 (local address 127.11.8.125:41549)
0806 15:02:51.490612 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:51.490614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:51.490717 (+ 103us) server_negotiation.cc:407] Connection header received
0806 15:02:51.490761 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:51.490762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:51.490773 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:51.490828 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:51.491453 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.491989 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.492909 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.493187 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.493231 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:51.502347 (+ 9116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:51.502349 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:51.502349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:51.502357 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:51.503930 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.504315 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.504316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.504316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.504375 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.504598 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.504599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.504599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.504724 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:51.504905 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:51.504944 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 15:02:51.505041 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 15:02:51.506824 11297 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 15:02:51.507740 10343 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51410
I20260806 15:02:51.507869 10343 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ea5cc7f1c4a4b6aaaa42d4da276b420 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 15:02:51.507936 10343 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ea5cc7f1c4a4b6aaaa42d4da276b420 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:51.508677 10343 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ea5cc7f1c4a4b6aaaa42d4da276b420 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44917 } } peers { permanent_uuid: "17d80ae85ffd48a596cfa7e588186df7" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 41549 } } peers { permanent_uuid: "63ab3db83fde447ea8a91f73c5fa23dc" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36887 } }
I20260806 15:02:51.508957 10343 leader_election.cc:288] T 00000000000000000000000000000000 P 7ea5cc7f1c4a4b6aaaa42d4da276b420 [CANDIDATE]: Term 1 election: Requested vote from peers 17d80ae85ffd48a596cfa7e588186df7 (127.11.8.125:41549), 63ab3db83fde447ea8a91f73c5fa23dc (127.11.8.124:36887)
I20260806 15:02:51.515193 10589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63ab3db83fde447ea8a91f73c5fa23dc: No bootstrap required, opened a new log
I20260806 15:02:51.517299 10589 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63ab3db83fde447ea8a91f73c5fa23dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44917 } } peers { permanent_uuid: "17d80ae85ffd48a596cfa7e588186df7" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 41549 } } peers { permanent_uuid: "63ab3db83fde447ea8a91f73c5fa23dc" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36887 } }
I20260806 15:02:51.517449 10589 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63ab3db83fde447ea8a91f73c5fa23dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63ab3db83fde447ea8a91f73c5fa23dc, State: Initialized, Role: FOLLOWER
I20260806 15:02:51.517530 10589 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63ab3db83fde447ea8a91f73c5fa23dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44917 } } peers { permanent_uuid: "17d80ae85ffd48a596cfa7e588186df7" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 41549 } } peers { permanent_uuid: "63ab3db83fde447ea8a91f73c5fa23dc" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36887 } }
I20260806 15:02:51.517854 10589 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63ab3db83fde447ea8a91f73c5fa23dc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:51.517956 10388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.510265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51420 (local address 127.11.8.126:44917)
0806 15:02:51.510302 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:51.510305 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:51.510355 (+ 50us) server_negotiation.cc:407] Connection header received
0806 15:02:51.510480 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:51.510481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:51.510493 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:51.510524 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:51.511325 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.512055 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.513044 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.513360 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.513421 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:51.514977 (+ 1556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:51.514978 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:51.514978 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:51.514986 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:51.516690 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.517086 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.517087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.517087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.517160 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.517359 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.517360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.517360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.517500 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:51.517605 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:51.517769 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 15:02:51.517855 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 15:02:51.517894 10630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63ab3db83fde447ea8a91f73c5fa23dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44917 } } peers { permanent_uuid: "17d80ae85ffd48a596cfa7e588186df7" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 41549 } } peers { permanent_uuid: "63ab3db83fde447ea8a91f73c5fa23dc" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36887 } } }
I20260806 15:02:51.518046 10630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63ab3db83fde447ea8a91f73c5fa23dc [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:02:51.518832 10558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "63ab3db83fde447ea8a91f73c5fa23dc"
I20260806 15:02:51.518945 10558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63ab3db83fde447ea8a91f73c5fa23dc [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:51.520210 10558 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 63ab3db83fde447ea8a91f73c5fa23dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ea5cc7f1c4a4b6aaaa42d4da276b420 in term 1.
I20260806 15:02:51.520442 10309 leader_election.cc:302] T 00000000000000000000000000000000 P 7ea5cc7f1c4a4b6aaaa42d4da276b420 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63ab3db83fde447ea8a91f73c5fa23dc, 7ea5cc7f1c4a4b6aaaa42d4da276b420; no voters:
I20260806 15:02:51.520592 10599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ea5cc7f1c4a4b6aaaa42d4da276b420 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 15:02:51.520754 10599 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ea5cc7f1c4a4b6aaaa42d4da276b420 [term 1 LEADER]: Becoming Leader. State: Replica: 7ea5cc7f1c4a4b6aaaa42d4da276b420, State: Running, Role: LEADER
I20260806 15:02:51.520874 10599 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ea5cc7f1c4a4b6aaaa42d4da276b420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44917 } } peers { permanent_uuid: "17d80ae85ffd48a596cfa7e588186df7" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 41549 } } peers { permanent_uuid: "63ab3db83fde447ea8a91f73c5fa23dc" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36887 } }
I20260806 15:02:51.521317 10599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ea5cc7f1c4a4b6aaaa42d4da276b420 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ea5cc7f1c4a4b6aaaa42d4da276b420. Latest consensus state: current_term: 1 leader_uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44917 } } peers { permanent_uuid: "17d80ae85ffd48a596cfa7e588186df7" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 41549 } } peers { permanent_uuid: "63ab3db83fde447ea8a91f73c5fa23dc" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36887 } } }
I20260806 15:02:51.521405 10599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ea5cc7f1c4a4b6aaaa42d4da276b420 [sys.catalog]: This master's current role is: LEADER
W20260806 15:02:51.523255 10652 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 63ab3db83fde447ea8a91f73c5fa23dc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:02:51.523773 10652 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:02:51.523933 10497 master_runner.cc:186] Error getting master registration for 127.11.8.125:41549: OK, instance_id { permanent_uuid: "17d80ae85ffd48a596cfa7e588186df7" instance_seqno: 1786028571303800 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 15:02:51.526067 10497 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 15:02:51.532544 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.510182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44550 (local address 127.11.8.125:41549)
0806 15:02:51.510231 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:51.510233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:51.521677 (+ 11444us) server_negotiation.cc:407] Connection header received
0806 15:02:51.521787 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:51.521789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:51.521800 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:51.521830 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:51.523867 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.524549 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.525525 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.525806 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.525850 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:51.527323 (+ 1473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:51.527325 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:51.527325 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:51.527332 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:51.529003 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.529426 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.529427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.529427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.529494 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.531894 (+ 2400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.531895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.531896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.532043 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:51.532153 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:51.532363 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 15:02:51.532449 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 15:02:51.533774 10597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.510167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56546 (local address 127.11.8.124:36887)
0806 15:02:51.510223 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:51.510225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:51.518331 (+ 8106us) server_negotiation.cc:407] Connection header received
0806 15:02:51.518474 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:51.518475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:51.518486 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:51.518518 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:51.519210 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.519686 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.521850 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.522145 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.522194 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:51.529709 (+ 7515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:51.529710 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:51.529711 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:51.529717 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:51.531481 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.532824 (+ 1343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.532825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.532826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.532892 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.533114 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.533116 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.533116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.533256 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:51.533336 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:51.533583 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 15:02:51.533682 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 15:02:51.534188 10654 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 15:02:51.534443 10654 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 15:02:51.536509 10558 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 63ab3db83fde447ea8a91f73c5fa23dc [term 1 FOLLOWER]: Refusing update from remote peer 7ea5cc7f1c4a4b6aaaa42d4da276b420: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:02:51.536898 10638 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7ea5cc7f1c4a4b6aaaa42d4da276b420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63ab3db83fde447ea8a91f73c5fa23dc" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:02:51.538381 10630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63ab3db83fde447ea8a91f73c5fa23dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ea5cc7f1c4a4b6aaaa42d4da276b420. Latest consensus state: current_term: 1 leader_uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44917 } } peers { permanent_uuid: "17d80ae85ffd48a596cfa7e588186df7" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 41549 } } peers { permanent_uuid: "63ab3db83fde447ea8a91f73c5fa23dc" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36887 } } }
I20260806 15:02:51.538512 10630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63ab3db83fde447ea8a91f73c5fa23dc [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:02:51.539536 10638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ea5cc7f1c4a4b6aaaa42d4da276b420 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44917 } } peers { permanent_uuid: "17d80ae85ffd48a596cfa7e588186df7" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 41549 } } peers { permanent_uuid: "63ab3db83fde447ea8a91f73c5fa23dc" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36887 } } }
I20260806 15:02:51.539623 10638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ea5cc7f1c4a4b6aaaa42d4da276b420 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:51.539767 10488 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36887 } attempt: 3
I20260806 15:02:51.540189 10630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63ab3db83fde447ea8a91f73c5fa23dc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44917 } } peers { permanent_uuid: "17d80ae85ffd48a596cfa7e588186df7" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 41549 } } peers { permanent_uuid: "63ab3db83fde447ea8a91f73c5fa23dc" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36887 } } }
I20260806 15:02:51.540266 10630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63ab3db83fde447ea8a91f73c5fa23dc [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:02:51.540505 10661 mvcc.cc:205] Tried to move back new op lower bound from 7315573029010100224 to 7315573028950261760. Current Snapshot: MvccSnapshot[applied={T|T < 7315573029010100224}]
I20260806 15:02:51.541102 10654 catalog_manager.cc:1347] Generated new cluster ID: 8a5559dde0e94ac7b1b122506e50add0
I20260806 15:02:51.541157 10654 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 15:02:51 dist-test-slave-68xk krb5kdc[10273](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786028571, etypes {rep=17 tkt=17 ses=17}, kudu/127.11.8.125@KRBTEST.COM for kudu/127.11.8.124@KRBTEST.COM
I20260806 15:02:51.548655 10491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.540026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.11.8.124:36887 (local address 127.0.0.1:56550)
0806 15:02:51.540215 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 15:02:51.540228 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 15:02:51.540270 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 15:02:51.540750 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 15:02:51.540751 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 15:02:51.540931 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 15:02:51.540977 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:51.540979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.541877 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.541878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:51.542720 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 15:02:51.542722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.543168 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.543169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 15:02:51.543224 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:51.543685 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 15:02:51.543690 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 15:02:51.545270 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 15:02:51.547410 (+ 2140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:51.547411 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:51.547411 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:51.547660 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:51.547931 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 15:02:51.547931 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 15:02:51.547932 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 15:02:51.548003 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 15:02:51.548336 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 15:02:51.548337 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 15:02:51.548396 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 15:02:51.548457 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 15:02:51.548555 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
I20260806 15:02:51.549185 10597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:51.540371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56550 (local address 127.11.8.124:36887)
0806 15:02:51.540485 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:51.540487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:51.540495 (+ 8us) server_negotiation.cc:407] Connection header received
0806 15:02:51.540514 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:51.540515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:51.540525 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:51.540579 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:51.541066 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.541803 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.542799 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:51.543087 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:51.543170 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:51.545382 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 15:02:51.545384 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 15:02:51.545384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 15:02:51.545391 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 15:02:51.547308 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.547772 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.547773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.547774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.547846 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 15:02:51.548082 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 15:02:51.548083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 15:02:51.548083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 15:02:51.548232 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 15:02:51.548291 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:51.548991 (+ 700us) server_negotiation.cc:299] Negotiation successful
0806 15:02:51.549084 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260806 15:02:51.550972 10488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17d80ae85ffd48a596cfa7e588186df7: Bootstrap starting.
I20260806 15:02:51.551785 10488 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17d80ae85ffd48a596cfa7e588186df7: Neither blocks nor log segments found. Creating new log.
I20260806 15:02:51.552078 10488 log.cc:824] T 00000000000000000000000000000000 P 17d80ae85ffd48a596cfa7e588186df7: Log is configured to *not* fsync() on all Append() calls
I20260806 15:02:51.553112 10488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17d80ae85ffd48a596cfa7e588186df7: No bootstrap required, opened a new log
I20260806 15:02:51.555688 10488 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17d80ae85ffd48a596cfa7e588186df7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44917 } } peers { permanent_uuid: "17d80ae85ffd48a596cfa7e588186df7" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 41549 } } peers { permanent_uuid: "63ab3db83fde447ea8a91f73c5fa23dc" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36887 } }
I20260806 15:02:51.555840 10488 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17d80ae85ffd48a596cfa7e588186df7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17d80ae85ffd48a596cfa7e588186df7, State: Initialized, Role: FOLLOWER
I20260806 15:02:51.555925 10488 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17d80ae85ffd48a596cfa7e588186df7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44917 } } peers { permanent_uuid: "17d80ae85ffd48a596cfa7e588186df7" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 41549 } } peers { permanent_uuid: "63ab3db83fde447ea8a91f73c5fa23dc" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36887 } }
I20260806 15:02:51.556267 10488 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17d80ae85ffd48a596cfa7e588186df7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 15:02:51.556288 10455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "17d80ae85ffd48a596cfa7e588186df7"
I20260806 15:02:51.556788 10455 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17d80ae85ffd48a596cfa7e588186df7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 15:02:51.557700 10455 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 17d80ae85ffd48a596cfa7e588186df7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ea5cc7f1c4a4b6aaaa42d4da276b420 in term 1.
I20260806 15:02:51.557984 10453 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 17d80ae85ffd48a596cfa7e588186df7 [term 1 FOLLOWER]: Refusing update from remote peer 7ea5cc7f1c4a4b6aaaa42d4da276b420: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 15:02:51.558498 10666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17d80ae85ffd48a596cfa7e588186df7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44917 } } peers { permanent_uuid: "17d80ae85ffd48a596cfa7e588186df7" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 41549 } } peers { permanent_uuid: "63ab3db83fde447ea8a91f73c5fa23dc" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36887 } } }
I20260806 15:02:51.558554 10599 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7ea5cc7f1c4a4b6aaaa42d4da276b420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17d80ae85ffd48a596cfa7e588186df7" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 41549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 15:02:51.558626 10666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17d80ae85ffd48a596cfa7e588186df7 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:02:51.560084 10666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17d80ae85ffd48a596cfa7e588186df7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ea5cc7f1c4a4b6aaaa42d4da276b420. Latest consensus state: current_term: 1 leader_uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44917 } } peers { permanent_uuid: "17d80ae85ffd48a596cfa7e588186df7" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 41549 } } peers { permanent_uuid: "63ab3db83fde447ea8a91f73c5fa23dc" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36887 } } }
I20260806 15:02:51.560245 10666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17d80ae85ffd48a596cfa7e588186df7 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:02:51.560325 10666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17d80ae85ffd48a596cfa7e588186df7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44917 } } peers { permanent_uuid: "17d80ae85ffd48a596cfa7e588186df7" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 41549 } } peers { permanent_uuid: "63ab3db83fde447ea8a91f73c5fa23dc" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36887 } } }
I20260806 15:02:51.560518 10666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17d80ae85ffd48a596cfa7e588186df7 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 15:02:51.562285 10654 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 15:02:51.563009 10654 catalog_manager.cc:1504] Loading token signing keys...
W20260806 15:02:51.567617 10683 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 17d80ae85ffd48a596cfa7e588186df7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 15:02:51.567701 10683 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 15:02:51.568096 10680 mvcc.cc:205] Tried to move back new op lower bound from 7315573029010100224 to 7315573028950261760. Current Snapshot: MvccSnapshot[applied={T|T < 7315573029010100224}]
I20260806 15:02:51.568886 10638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ea5cc7f1c4a4b6aaaa42d4da276b420 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ea5cc7f1c4a4b6aaaa42d4da276b420" member_type: VOTER last_known_addr { host: "127.11.8.126" port: 44917 } } peers { permanent_uuid: "17d80ae85ffd48a596cfa7e588186df7" member_type: VOTER last_known_addr { host: "127.11.8.125" port: 41549 } } peers { permanent_uuid: "63ab3db83fde447ea8a91f73c5fa23dc" member_type: VOTER last_known_addr { host: "127.11.8.124" port: 36887 } } }
I20260806 15:02:51.568979 10638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ea5cc7f1c4a4b6aaaa42d4da276b420 [sys.catalog]: This master's current role is: LEADER
I20260806 15:02:51.577087 10654 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ea5cc7f1c4a4b6aaaa42d4da276b420: Generated new TSK 0
I20260806 15:02:51.577255 10654 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 15:02:51.608727 10494 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:02:51.606744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44560 (local address 127.11.8.125:41549)
0806 15:02:51.606791 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:51.606793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:51.607815 (+ 1022us) server_negotiation.cc:407] Connection header received
0806 15:02:51.608060 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:51.608061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:51.608073 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:51.608105 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:51.608657 (+ 552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44560: BlockingRecv error: recv got EOF from 127.0.0.1:44560 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260806 15:02:51.609313 10597 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:02:51.606966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56556 (local address 127.11.8.124:36887)
0806 15:02:51.607006 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:51.607008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:51.607728 (+ 720us) server_negotiation.cc:407] Connection header received
0806 15:02:51.608175 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:51.608176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:51.608187 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:51.608216 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:51.609250 (+ 1034us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56556: BlockingRecv error: recv got EOF from 127.0.0.1:56556 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 15:02:51.609974 10388 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:02:51.607177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51430 (local address 127.11.8.126:44917)
0806 15:02:51.607217 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:51.607219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:51.607956 (+ 737us) server_negotiation.cc:407] Connection header received
0806 15:02:51.609473 (+ 1517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:51.609475 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:51.609487 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:51.609532 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:51.609911 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51430: BlockingRecv error: recv got EOF from 127.0.0.1:51430 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 15:02:51.612407 10494 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:02:51.611722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44564 (local address 127.11.8.125:41549)
0806 15:02:51.611756 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:51.611757 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:51.611929 (+ 172us) server_negotiation.cc:407] Connection header received
0806 15:02:51.612031 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:51.612032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:51.612050 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:51.612079 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:51.612361 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44564: BlockingRecv error: recv got EOF from 127.0.0.1:44564 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 15:02:51.615443 10494 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:02:51.614711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44580 (local address 127.11.8.125:41549)
0806 15:02:51.614744 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:51.614746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:51.614946 (+ 200us) server_negotiation.cc:407] Connection header received
0806 15:02:51.615036 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:51.615037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:51.615046 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:51.615073 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:51.615401 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44580: BlockingRecv error: recv got EOF from 127.0.0.1:44580 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 15:02:51.623091 10494 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:02:51.618593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44586 (local address 127.11.8.125:41549)
0806 15:02:51.618624 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:51.618637 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:51.622599 (+ 3962us) server_negotiation.cc:407] Connection header received
0806 15:02:51.622709 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:51.622710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:51.622720 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:51.622748 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:51.623048 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44586: BlockingRecv error: recv got EOF from 127.0.0.1:44586 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 15:02:51.628868 10494 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:02:51.628122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44600 (local address 127.11.8.125:41549)
0806 15:02:51.628156 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:51.628158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:51.628357 (+ 199us) server_negotiation.cc:407] Connection header received
0806 15:02:51.628447 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:51.628448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:51.628457 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:51.628485 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:51.628824 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44600: BlockingRecv error: recv got EOF from 127.0.0.1:44600 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 15:02:51.638826 10494 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:02:51.638026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44604 (local address 127.11.8.125:41549)
0806 15:02:51.638061 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:51.638063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:51.638270 (+ 207us) server_negotiation.cc:407] Connection header received
0806 15:02:51.638363 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:51.638364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:51.638374 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:51.638403 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:51.638779 (+ 376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44604: BlockingRecv error: recv got EOF from 127.0.0.1:44604 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 15:02:51.657066 10494 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:02:51.656177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44618 (local address 127.11.8.125:41549)
0806 15:02:51.656219 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:51.656221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:51.656475 (+ 254us) server_negotiation.cc:407] Connection header received
0806 15:02:51.656582 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:51.656584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:51.656595 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:51.656624 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:51.657010 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44618: BlockingRecv error: recv got EOF from 127.0.0.1:44618 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 15:02:51.691428 10494 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:02:51.690554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44622 (local address 127.11.8.125:41549)
0806 15:02:51.690602 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:51.690604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:51.690802 (+ 198us) server_negotiation.cc:407] Connection header received
0806 15:02:51.690843 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:51.690844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:51.690859 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:51.690934 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:51.691363 (+ 429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44622: BlockingRecv error: recv got EOF from 127.0.0.1:44622 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260806 15:02:51.757486 10494 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:02:51.756583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44638 (local address 127.11.8.125:41549)
0806 15:02:51.756680 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:51.756682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:51.756763 (+ 81us) server_negotiation.cc:407] Connection header received
0806 15:02:51.756900 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:51.756902 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:51.756915 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:51.756944 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:51.757424 (+ 480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44638: BlockingRecv error: recv got EOF from 127.0.0.1:44638 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260806 15:02:51.887570 10494 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:02:51.886804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44646 (local address 127.11.8.125:41549)
0806 15:02:51.886892 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:51.886894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:51.886946 (+ 52us) server_negotiation.cc:407] Connection header received
0806 15:02:51.887012 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:51.887013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:51.887023 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:51.887067 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:51.887476 (+ 409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44646: BlockingRecv error: recv got EOF from 127.0.0.1:44646 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260806 15:02:52.145557 10494 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 15:02:52.144825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44648 (local address 127.11.8.125:41549)
0806 15:02:52.144924 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:52.144926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:52.144939 (+ 13us) server_negotiation.cc:407] Connection header received
0806 15:02:52.144996 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:52.144997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:52.145008 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:52.145092 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 15:02:52.145507 (+ 415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44648: BlockingRecv error: recv got EOF from 127.0.0.1:44648 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 15:02:52.524250 10652 catalog_manager.cc:1259] Loaded cluster ID: 8a5559dde0e94ac7b1b122506e50add0
I20260806 15:02:52.524362 10652 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 63ab3db83fde447ea8a91f73c5fa23dc: loading cluster ID for follower catalog manager: success
I20260806 15:02:52.525199 10652 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 63ab3db83fde447ea8a91f73c5fa23dc: acquiring CA information for follower catalog manager: success
I20260806 15:02:52.525528 10652 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 63ab3db83fde447ea8a91f73c5fa23dc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 15:02:52.568192 10683 catalog_manager.cc:1259] Loaded cluster ID: 8a5559dde0e94ac7b1b122506e50add0
I20260806 15:02:52.568328 10683 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 17d80ae85ffd48a596cfa7e588186df7: loading cluster ID for follower catalog manager: success
I20260806 15:02:52.569144 10683 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 17d80ae85ffd48a596cfa7e588186df7: acquiring CA information for follower catalog manager: success
I20260806 15:02:52.569481 10683 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 17d80ae85ffd48a596cfa7e588186df7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 15:02:52.661692 10764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:52.658993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44664 (local address 127.11.8.125:41549)
0806 15:02:52.659115 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:52.659117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:52.659184 (+ 67us) server_negotiation.cc:407] Connection header received
0806 15:02:52.659245 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:52.659246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:52.659300 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:52.659387 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:02:52.659534 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:52.660019 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:52.660854 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:52.661101 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:52.661160 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:52.661326 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:02:52.661437 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:02:52.661498 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:52.661608 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 15:02:52.661625 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":73,"threads_started":1}
I20260806 15:02:52.665874 10772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:52.663573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56566 (local address 127.11.8.124:36887)
0806 15:02:52.663678 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:52.663680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:52.663690 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:02:52.663736 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:52.663737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:52.663774 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:52.663851 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:02:52.663924 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:52.664442 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:52.665174 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:52.665382 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:52.665436 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:52.665548 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:02:52.665648 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:02:52.665704 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:52.665790 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 15:02:52.665807 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":56,"threads_started":1}
I20260806 15:02:52.670387 10772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:52.667649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56576 (local address 127.11.8.124:36887)
0806 15:02:52.667693 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:52.667696 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:52.667945 (+ 249us) server_negotiation.cc:407] Connection header received
0806 15:02:52.667971 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:52.667972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:52.667985 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:52.668037 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:02:52.668251 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:52.668954 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:52.669661 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:52.669871 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:52.669946 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:52.670107 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:02:52.670137 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:02:52.670189 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:52.670276 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 15:02:52.670297 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 15:02:52.670936 10764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:52.667725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44676 (local address 127.11.8.125:41549)
0806 15:02:52.667772 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:52.667775 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:52.667885 (+ 110us) server_negotiation.cc:407] Connection header received
0806 15:02:52.667928 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:52.667929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:52.667942 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:52.667997 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:02:52.668143 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:52.668877 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:52.669987 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:52.670303 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:52.670346 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:52.670632 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:02:52.670666 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:02:52.670696 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:52.670828 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 15:02:52.670856 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 15:02:52.671057 10789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 15:02:52.667614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51440 (local address 127.11.8.126:44917)
0806 15:02:52.667790 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 15:02:52.667792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 15:02:52.667802 (+ 10us) server_negotiation.cc:407] Connection header received
0806 15:02:52.667849 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 15:02:52.667850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 15:02:52.667902 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 15:02:52.667982 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 15:02:52.668089 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:52.668759 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:52.669980 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 15:02:52.670340 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 15:02:52.670403 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 15:02:52.670660 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 15:02:52.670775 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 15:02:52.670845 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 15:02:52.670910 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 15:02:52.670940 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":65,"threads_started":1}
I20260806 15:02:52.672907 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 10287
I20260806 15:02:52.684657 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 10392
I20260806 15:02:52.694725 11297 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskF1cjFt/build/release/bin/kudu with pid 10497
2026-08-06T15:02:52Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3021 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6143 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (284956 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST